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The  objective  of  our  efforts  was  to  extend  and  apply  a  new  adaptive  control  technique 
based  on  a  disturbance  attenuation  bound.  The  structure  of  this  new  adaptive  control 
scheme  is  the  result  of  formulating  a  disturbance  attenuation  problem  for  a  particular  class  of 
nonlinear  systems  whose  solution  is  obtained  without  any  approximation.  A  global  solution 
is  obtained  and  must  be  contrasted  with  much  of  the  nonlinear  Woo  results  which  assume 
that  the  scheme  operates  locally  about  some  equilibrium  condition. 

The  class  of  nonlinearities  considered  is  that  of  a  linear  system  where  the  coefficient 
matrix  of  the  control  is  assumed  to  be  a  linear  function  of  an  unknown  parameter.  The  work 
performed  on  this  grant  extended  this  class  to  include  state  coefficients  matrices  linear  in 
the  parameter  if  the  associated  state  that  multiplies  this  term  is  measured  perfectly. 

.  To  bring  these  mathematical  abstractions  to  engineering  practice,  a  significant  effort  was 
made  to  apply  this  new  adaptive  control  scheme  to  the  development  of  an  adaptive  flight 
control  system  for  a  high  angle-of-attack  aircraft  such  as  the  F-18  HARV  (High  Angle-of- 
attack  Research  Vehicle).  We  are  just  beginning  to  show  performance  improvements  in  the 
time  response  over  that  of  standard  adaptive  controllers  due  to  an  initial  reduction  in  the 
control  effort  associated  with  those  control  system  parameters  that  are  initially  uncertain. 
This  new  adaptive  controller  involves  not  only  the  state  and  parameter  estimates,  but  also 
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the  pseudo  covariance  matrix.  This  new  adaptive  scheme  does  require  the  determination  of 
the  global  maxima  of  a  certain  function  with  respect  to  the  uncertain  parameters. 

Our  current  results  are  for  a  single  parameter.  As  in  the  F-111  adaptive  flight  control 
system,  we  focus  on  estimating  the  moment  coefficient  due  to  elevator  deflection.  Our 
goals  are  to  extend  this  to  multiple  parameters  and  to  eliminate  the  restriction  on  perfect 
information  so  that  all  parameters  in  the  system  can  be  included.  Our  approach  to  removing 
this  restriction  is  to  use  perturbation  theory  associated  with  an  assumed  small  measurement 
uncertainty  weighting  matrix.  Complementary  results  using  perturbation  theory  are  being 
pursued  on  our  main  AFOSR  grant  (F49620-91-0077). 

1  Introduction 

Current  adaptive  control  schemes  assume  certain  equivalence.  That  is,  the  structure  of 
the  adaptive  controller  assumes  a  parameter  identifier  in  cascade  with  a  controller  such  as 
the  linear-quadratic-Gaussian  (LQG)  controller  or  multi-step  predicted  output  control.  Al¬ 
though  the  proper  approach  to  adaptive  control  is  based  on  stochastic  control  theory,  it  is 
untractable.  Even  the  simplest  extension  of  the  LQG  problem  is  unmechanizable.  This  ex¬ 
tension  involves  a  linear  system  where  the  control  coefficient  matrix  is  a  linear  function  of  an 
unknown  parameter.  If  this  parameter  set  is  augmented  to  the  original  state  space,  a  finite  di¬ 
mensional  conditional  Gaussian  estimator  is  used  to  reduce  this  problem  to  a  full  information 
problem.  Nevertheless,  the  resulting  stochastic  control  problem  of  minimizing  the  expected 
value  of  a  quadratic  performance  index  is  subject  to  a  stochastic  vector  differential  equation 
and  a  Riccati  differential  equation.  To  date  this  problem  remains  untractable  except  for 
perturbation  methods  based  on  small  measurement  noise  spectral  densities.  This  approach 
is  showing  significant  improvement  and  is  being  developed  under  our  current  AFOSR  grant. 

An  alternate,  but  deterministic,  approach  is  formulated  by  determining  a  controller  which 
bounds  a  disturbance  attenuation  function  against  all  admissible  measurement  and  process 
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disturbances  and  initial  conditions.  For  the  stochastic  control  problem  described  above,  i.e. 
the  control  coefficient  is  a  linear  function  of  a  parameter  vector,  a  disturbance  attenuation 
controller  can  be  found  via  a  dynamic  programming  solution  [1].  The  problem  is  shown  in 
[1]  to  decompose  into  a  controller  problem  with  full  information  with  an  associated  optimal 
return  function  representing  the  optimal  cost  into  the  future  and  an  estimation  problem 
with  an  associated  optimal  accumulation  function  representing  the  optimal  cost  due  to  the 
control  and  disturbances  from  the  past.  The  maximum  of  the  sum  of  these  two  functions 
with  respect  to  the  unknown  current  state  produces  the  worst  case  state.  A  sufficiency 
theorem  in  [2]  requires  that  for  a  saddle  point  controller  to  exist  the  worst  case  state  must 
be  a  unique  global  maximum.  For  this  class  of  disturbance  attenuation  problem,  the  global 
maximum  are  shown  to  be  nonunique.  However,  in  [1]  it  is  shown  that  the  resulting  control 
strategy  is  still  a  saddle  point  strategy  since  it  is  proven  that  the  control  strategy  when  there 
is  not  a  unique  global  worst  case  state  is  unique.  An  alternate  and  direct  proof  is  given  in 
[3]  where  the  infinite-time  results  are  also  presented. 

This  powerful  result  forms  the  basis  of  a  new  approach  to  adaptive  control.  In  the  next 
section  we  describe  the  work  performed  on  this  supplemental  grant.  In  the  following  section 
we  describe  the  work  that  would  have  been  performed  if  the  no  cost  extension  had  been 
allowed. 

2  An  Extension  of  of  the  Adaptive  Controller  Based 
on  Disturbance  Attenuation  with  Application  to  Air¬ 
craft  Flight  Control 

In  the  disturbance  attenuation  adaptive  controller  of  [1,3]  only  the  control  coefficient  matrix 
is  a  linear  function  of  an  uncertain  set  of  parameters.  This  new  adaptive  control  technique 
is  extended  to  include  some  of  the  parameters  in  the  state  coefficient  matrix.  To  ensure 
that  the  estimator  remains  finite  dimensional,  only  the  parameters  of  the  state  coefficient 
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matrix  which  multiplies  a  state  element  that  is  perfectly  measured  are  used  in  the  extension 
of  the  new  adaptive  control  law.  This  generalization  allows  some  reduction  in  the  estimator 
dimension,  but  the  worst  case  state  is  produced  now  from  maximizing  the  sum  of  optimal 
return  function  and  the  optimal  accumulation  function  subject  to  a  constraint  formed  by  the 
perfect  measurements.  Details  of  this  analysis  are  given  in  Appendix  A.  In  the  next  section, 
we  discuss  further  generalization  to  the  partial  information  case  by  perturbation  methods. 

This  research  supplement  is  motivated  by  the  need  to  learn  how  to  implement  this  adap¬ 
tive  controller  in  important  applications  such  as  aircraft  flight  control  systems.  To  evaluate 
controller  design  a  high  fidelity  nonlinear  simulation  of  the  F-18  high  angle-of-attack  research 
vehicle  (HARV)  was  obtained  from  NASA  Dryden.  A  parameter-robust  game  theoretic  com¬ 
pensator  was  designed  to  track  pilot  inputs  in  angle-of-attack,  sideslip,  and  stability-axis  roll 
rate  through  stick  and  rudder  pedal  commands.  Zero  steady  state  error  in  the  presence  of 
step  inputs  is  achieved  by  using  a  system  of  error  coordinates  which  also  allows  for  thrust 
vector  commands  to  fade  to  zero  in  steady  state.  Results  of  this  study  are  given  in  Appendix 
B.  This  controller  and  its  performance  were  to  be  used  as  a  benchmark  to  compare  the  results 
of  the  adaptive  controller. 

To  begin  to  understand  the  implementation  issues  of  this  new  adaptive  controller  the 
longitudinal  mode  of  the  F-18  HARV  was  first  controlled.  Parameter  uncertainty  in  both  the 
state  and  control  matrices  were  considered.  In  particular,  stability  derivatives  that  multiply 
the  inertial  states  were  used  to  augment  the  state  vector  to  be  estimated  on-line.  Since 
four  unknown  parameters  were  included  in  the  adaptive  controller,  a  search  for  the  global 
maximum  of  the  sum  of  the  optimal  return  function  and  optimal  accumulation  function  with 
respect  to  these  four  parameters  was  not  attempted.  Rather  a  possible  local  worst  case  state 
is  obtained  near  the  estimated  state.  The  performance  of  the  new  adaptive  controller  was 
similar  to  that  of  standard  adaptive  controllers  assuming  certainty  equivalence.  It  appears 
that  this  approach  to  adaptive  control  is  equivalent  to  nonlinear  Hoo  controllers  which  assume 
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local  behavior  near  an  equilibrium  point. 

To  obtain  dramatic  performance,  it  appeared  that  the  global  aspects  of  the  controller 
was  to  be  explored.  The  parameter  set  was  reduced  to  a  scalar,  the  moment  coefficient  due 
to  elevator  deflection.  For  certain  initial  conditions  remarkable  performance  is  obtained  over 
current  adaptive  controllers.  In  regimes  where  the  scalar  parameter  is  quite  unknown  the 
control  emphasizes  thrust  vectoring  and  reduces  the  elevator  deflection  to  be  almost  zero.  As 
more  information  is  obtained  the  controller  begins  to  use  more  elevator  deflection  and  less 
thrust  vectoring  which  is  eventually  faded  to  zero.  This  is  to  be  contrasted  with  standard 
adaptive  controllers  in  which  substantial  elevator  deflection  is  used  early  even  though  the 
parameter  is  the  wrong  sign.  Therefore,  the  initial  response  is  in  the  wrong  direction.  The 
inherent  conservative,  but  intelligent,  performance  of  the  new  controller  is  associated  with 
the  two  worst  case  states  in  which  usually  only  one  is  a  global  maximum.  In  the  beginning 
the  global  worst  case  state  dictates  a  conservative  policy  where  the  elevator  deflection  is 
made  small  and  the  thrust  vectoring  dominates  the  response.  At  some  time,  say  4,  both 
worst  case  states  produce  identical  cost.  Here,  there  is  a  switch  from  the  conservative  policy 
to  one  similar  to  that  of  the  standard  adaptive  controller.  Note  that  the  controls  do  remain 
continuous,  even  at  tc-  Our  current  study  evaluating  the  performance  of  this  new  adaptive 
controller  for  flight  control  is  described  in  detail  in  Appendix  C. 

3  Future  Work  in  the  Adaptive  Disturbance  Attenua¬ 
tion  Controller  and  It’s  Application 

Based  upon  the  encouraging  performance  of  the  new  adaptive  flight  controller  described 
in  Appendix  C,  the  following  research  directions  are  proposed.  To  generalize  the  current 
procedure  to  include  partial  information  and  parameter  uncertainty  in  both  the  state  and 
control  coefficient  matrices,  a  perturbation  methodology  is  suggested.  Here,  two  approaches 
are  possible.  Either  the  measurement  uncertainty  weighting  in  the  disturbance  attenuation 
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function  could  be  assumed  small  and,  therefore,  an  expansion  parameter,  or  the  coefficient 
of  the  parameter  in  the  state  coefficient  matrix  could  be  assumed  small  and,  therefore,  an 
expansion  parameter.  This  second  approach  in  a  more  general  setting  is  being  explored  in 
our  main  current  AFOSR  grant.  Our  current  results  are  given  in  Appendix  D.  In  either 
approach  our  current  results  [l,3,Appendix  A]  form  the  zeroth-order  solution.  As  shown  in 
Appendix  D,  additional  higher-order  terms  remain  finite-dimensional. 

Application  of  this  new  controller  has  been  shown  to  produce  dramatic  and  intelligent 
performance  for  one  parameter.  Efficient  numerical  methods  are  being  sought  which  not 
only  produce  the  global  maximum,  but  keeps  track  of  all  the  local  maximum  to  determine 
when  one  maximum  value  moves  past  another  inducing  a  change  in  strategy,  but  keeping 
the  control  continuous. 

As  shown  in  Appendix  C,  the  new  adaptive  controller  is  a  rather  cautious  controller. 
When  the  uncertainty  in  the  coefficient  of  the  elevator  deflection  is  high,  emphasis  is  placed 
on  the  thrust  vectoring.  When  the  parameter  becomes  better  known,  the  elevator  is  used 
more,  and  in  steady  state  the  thrust  vectoring  is  faded  out.  This  is  required  due  to  the 
deterioration  of  the  thrust  vectoring  paddles  used  on  the  F-18  HARV.  The  strategy  of  this 
new  adaptive  control  law  is  highly  governed  by  not  only  the  state  and  parametric  estimates, 
but  their  associated  pseudo  error  variance.  This  is  unique  among  implementable  adaptive 
controllers.  Therefore,  our  future  efforts  are  directed  to  understanding  the  behavior  of  this 
new  adaptive  technique  through  examples  such  as  the  adaptive  flight  controller  of  the  F-18 
HARV,  extensions  to  multiple  uncertain  parameters,  and  extensions  to  the  theory  to  include 
constraints  on  the  parameter  uncertainties.  This  last  item  has  the  difficulty  that  we  wish 
to  preserve  the  saddle  point  structure  since  the  controller  is,  in  a  sense,  certainly  equivalent 
and  therefore,  easier  to  compute  on-line.  Otherwise,  one  is  left  with  the  difficult  problem  of 
finding  a  minimax  controller. 
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4  Conclusion 
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This  supplement  to  our  AFOSR  grant  has  been  important  in  bring  to  practice  an  impor¬ 
tant  new  adaptive  control  law  conceived  and  developed  from  a  theoretical  viewpoint  under 
AFOSR  sponsorship.  Charles  Dhillon  brings  to  this  research  project  a  strong  background 
and  interest  in  the  underlying  flight  control  problem  and  in  the  application  and  extension 
of  this  new  adaptive  control  law  to  aircraft  and  missile  flight  control  systems.  Our  progress 
has  not  been  as  rapid  as  desirable,  but  we  have  kept  our  sight  squarely  on  the  need  to  bring 
important  theory  to  engineering  practice,  an  activity  usually  overlooked. 
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Abstract 

The  i>uri>ose  of  tJjis  paper  is  to  present  extensions  to  robust  and  adap¬ 
tive  control  design  techniques  based  on  disturbance  attenuation  to  a  class 
of  systems  which  have  a  portion  of  the  state  space  which  is  measured  per¬ 
fectly  (without  juldit  ive  noise)  in  addition  to  a  set  of  measurements  which 
are  corrujited  by  noise.  For  the  robust  controller,  a  system  results  which 
is  similar  to  that  obtained  from  a  Parameter.Robust  Game  Theoretic  Syn¬ 
thesis  (PR GTS),  with  a  reduction  in  tlie  order  of  the  state  estimator.  In 
the  adaptive  case,  the  techniques  presented  allow  for  the  extension  of  the 
estimator  to  include  plant  state  coefficients  which  multiply  states  which 
are  measured  perfectly. 


Keywords:  Robust  Adaptive  Control,  Disturbance  Attenuation,  Dy¬ 
namic  Programming. 


1  Introduction 

Several  approadies  to  controlling  dynamic  systems  with  uncertain  parameters 
have  been  developed  in  recent  years  (1,  2,  3,  4].The  work  presented  in  this  paper 
provides  extensions  to  two  sucli  approaches  to  control  of  uncertain  systems  to 
a  class  of  systems  which  (xintains  a  combination  of  noisy  and  perfect  measure¬ 
ments.  Fii-st,  a  comi>ensator  whidi  is  robust  to  dianges  in  system  parameters  is 
developed  for  tliis  class  of  systems  in  a  manner  similar  to  [2, 4)  whidi  results  in  a 
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reduced  order  robust  compensator.  Secondly,  the  work  of  [1]  is  extended  to  this 
class  of  systevns,  whicJi  results  in  a  compensator  which  is  adaptive  to  uncertain 
parameter  multiplying  perfectly  measured  states  as  well  as  the  controls. 

In  [4],  a  controller  was  developed  using  a  game  theoretical  approach  to  the 
solution  of  a  disturbance  attenuation  problem  which,  when  combined  with  the 
use  of  an  internal  feedback  loop  (IFL)  decomi>osition  to  represent  the  uncertain- 
ties  in  the  coefficients  of  the  linear  s>'stem,  essentially  produced  a  control  which 
is  robust  to  the  parameter  variations  within  the  model.  By  extending  these 
results  to  the  class  of  systems  considered  in  this  paper,  the  order  of  the  com¬ 
pensator  is  reduced  by  considering  the  perfectly  measured  states  in  a  manner 
similar  to  that  developed  in  (5]. 

In  (1],  a  similar  disturbance,  attenuation  ])roblein  was  formulated,  but  with  an 
augmented  system  which  included  parameter  uncertainties  in  coefficients  mul¬ 
tiplying  the  controls.  The  disturbance  attenuation  problem  for  the  augmented 
system  was  then  solved  asing  a  dynamic  programming  approadi,  similar  to  that 
described  in  [2]  to  yield  an  adaptive  comi>ensator  structure  which  included  an 
estimator  for  determining  values  of  the  state  and  control  coefficient  parameters, 
a  controller  as  a  function  of  the  state  but  dei)endent  on  the  parameter  values, 
and  a  connection  con<lition  whicJi  linked  the  two.  In  the  connection  condition, 
the  sum  of  the  optimal  return  function  associated  with  the  control  problem  and 
the  ojjtimal  accumulation  function  a.ssociated  with  the  estimation  problem  re¬ 
sulting  from  the  dectomposition  of  the  dynamic  programming  solution  process  is 
maximized  with  re.sj)ect  to  the  current  state,  i.e.  the  worst  case  state.  For  this 
class  of  adaptive  control  problems,  in  contrast  to  the  results  in  (2,  3],  the  adap¬ 
tive  compensator  structure  is  based  on  the  results  of  {!]  where  the  restrictive 
assumption  that  the  worst  case  state  be  a  singleton  is  not  required. 

In  both  of  these  approaches  (1,  4],  it  was  assumed  that  all  measurements 
recteived  were  corrupted  by  an  external  disturbance.  In  some  systems,  however, 
it  may  arise  that  a  subset  of  the  measurements  may  be  known  perfectly,  (or  at 
least  clo.se  enough  to  be  modeled  as  such).  One  such  case  could  be  the  dxTiamics 
of  an  airplane,  where  body  angular  rates  are  measured  very  accurately,  but 
measurements  of  state  variables  such  as  angle-of-attack  and  sideslip  are  not 
nearly  a.s  e^Lsily  or  precisely  measurerJ. 

In  such  a  case,  wliere  a  portion  of  the  measurements  may  contain  noise 
while  another  does  not,  it  is  possible  to  extend  the  methods  described  aliove.  In 
the  case  of  the  robust  controller,  by  assuming  certain  measurements  are  known 
exactly,  a  cximpensator  of  reduced  order  may  be  designed  for  the  system.  In  the 
case  of  the  adaptive  approach,  an  additional  set  of  parameters,  those  multiplying 
the  states  which  are  measured  perfectly,  can  be  included  in  the  augmented  state 
vector  and  sul>sequently  be  estimated  on-line.  The  results  presented  in  this 
paper  will  provide  detail  on  these  extensions. 


2 


2  Djntiamic  System 

The  class  of  system  under  consideration  is  a  linear  system  with  uncertain  co¬ 
efficients  multiplying  states  and  controls  and  state  dynamics  which  are  forced 
by  a  random  disturbance.  Additionally,  the  class  of  systems  considered  has  a 
subset  of  the  state  space  which  is  measured  perfectly  in  addition  to  a  set  of 
measurements  which  are  corrupted  by  noise.  This  system  can  be  written  in  the 
following  fonn: 


X 

=  A(a)x  B  (/3)ii-hrw 

(1) 

=  Hix-hv 

(2) 

^2 

=  Bsx 

(3) 

Where  a  e  and  are  used  to  designate  uncertain  parameters  in  the 

state  and  control  coefficient  matrices. 

Two  ai^proaclies  will  l>e  considered  in  determining  a  control  strategy  for  this 
system.  The  fii-st  approach  is  to  design  the  controller  so  that  it  is  robust  to  vari¬ 
ations  in  the  j>aranieters  represented  by  q  and  P  by  using  an  intenial  feedback 
looj)  (IFL)  decoinjjosition  to  include  the  parameter  uncertainty  in  the  design 
process,  so  that  the  resulting  system  is  in  a  form  consistent  with  the  approach 
of  [4].  Secondly,  a  controller  is  considered  which  is  adajjtlve  to  the  parameters 
Q  and  /?.  In  order  to  pose  the  problem  in  a  manner  whidi  is  consistent  with  the 
theory  for  each  apj^roach,  we  can  first  rewrite  the  dynamic  equations  of  motion 
in  a  form  more  familiar  to  each  method. 

2.1  Dynamic  System  for  Reduced  Order  Robust  Controller 

In  the  c.ase  of  the  robust  aj^jn’oach,  we  can  write  the  system  state  and  control 
coefficient  matrices,  A  anrl  B,  in  terms  of  a  nominal  j)lant  Ao  and  Bo,  and 
j)arameter  dependent  j>erturbation  AA  and  AB: 


x  =  (Ao  +  AA  (o)).T -f  (Bo  +  AB  {/9))u -h  Fw;  (4) 


where 


AA  =  DLa(a)E 
AB  =  FLi{0)G 

Using  an  IFL  decomposition  of  AA  and  AB,  we  include  the  parameter  uncer¬ 
tainties  as  a  fictitious  disturbance,  so  that  the  system  which  we  will  be  analyzing 
has  the  standard  form: 


X  =  Ao-t  4-  Bow  +  fw; 


(S) 
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where  w  includes  external  disturbances  as  well  as  the  fictitious  disturbances 
introduced  by  the  decomposition  of  the  parameter  uncertainties  and  f  reflects 
how  these  disturbances  arc  introduced  into  the  system. 

2.2  Dynamic  System  for  Adaptive  Controller  with  Some 
State  Coefficients  Estimated 

In  order  to  apply  and  extend  the  theory  developed  in  [1]  to  the  class  of  systems 
being  considered,  we  first  must  write  the  dynamic  equations  of  motion  in  a  form 
which  is  consistent  with  its  development.  To  do  this,  we  can  apply  a  similarity 
transformation  to  the  original  system,  if  necessary,  to  obtain  a  system  which  is 
partitioned  into  the  states  which  are  included  in  the  set  of  perfect  measurements 
and  those  which  are  not.  This  system  can  then  be  written  in  the  form: 


where,  the  parameter  uncertainties  in  the  matrices  A\2  (o),  ^422  («)»  Bi  (0)  and 
B2  (0)  include  parameter  uncertainties  in  the  form: 

k 

Ai2{o)  =  (>^12)0  d-  y^Mi2)jQi 

k 

A22M  =  (>122)0  +  y^(^22)jQi 
/ 

B^{0)  =  + 

i 

B2{0)  =  (B2)o  +  Yl^B2h0j 


We  tlien  csiii  form  nn  augmented  state  vector,  $  = 
dynamics  are  defined  by  the  sj-stem: 

[  xjx^a^0^  ]^,  whose 

4  +  BfU  +  r^ti; 

(12) 

=  Hi^  +  w 

(13) 

Z2  =  ^2$ 

(14) 

where 


(8) 

(9) 

(10) 

(11) 


■^11  (Ai7)o  I  (Al2)lZ2  •••  {Al2)kZ2  >  I  {Bi)iU  •••  {Bi)lU 

_  A21  {A22)o  {  {A22)\Z2  •••  (A22)kZ2  }  {  •••  (BajjU 

®  0  0  0 

.0  0  0  0 

and 


=  [  i/3  H4  00]  ^2 = [  0  /  0  0  ] 

In  tliis  fonmiiation,  the  perfect  ineasureinents  xj  act  as  an  additional  known 
input  to  tlie  system.  TIiils,  they  could  effectively  be  considered  as  part  of  an 
augmented  injmt  vector  it,  such  that  the  parameter  de]>endence  is  then  only 
in  the  augmented  ini)ut  matrix.  This  then  generalizes  the  class  of  systems 
ctMisidererl  in  (1]  and  is  consistent  with  the  theory  presented  therein. 

3  The  Disturbance  Attenuation  Problem 

For  this  system,  a  disturbance  attenuation  function  is  formed  as  in  (1],  which  is 
e~ssentially  the  rati«)  of  norms  of  perfonnance  outputs  over  disturbance  inputs. 
The  problem  can  be  written  as; 

e>0  (15) 

where  the  inesusures  of  i>erformance  outputs,  ||j/||2  and  |1«;||^  are  defined  as 

ll?/ll^  =  (INIq  +  ll»‘lln) ‘I’’  (10) 

Mf  =  lia«)||p-.+^'(lHI?v-.  +  IHI^-.)fir  (17) 

where,  in  the  case  of  the  non-augmentefl  system  for  the  robust  controller,  f  is 
simply  X. 

This  disturbance  attenuation  function  can  then  be  converted  to  a  perfor¬ 
mance  index,  given  by  (18),  which  gives  risse  to  a  differential  game  problem. 


The  basic  idea,  then,  is  to  find  the  cx)ntrol,  u,  which  minimizes  the  distur¬ 
bance  attenuation  function  subject  to  tlie  worst  case  maximizing  disturbance 
inputs  provided  by  initial  conditions,  x(0),  parameter  uncertainty  (a,^),  state 
noise  iw,  and  measurement  noise  v. 


4  Dynamic  Programming  Solution 

The  approach  taken  in  finding  the  solution  to  the  disturbance  attenuation  prob¬ 
lem  is  to  use  a  dynamic  programming  technique  to  separate  the  problem  into 
two  sej>arate  ]>roblems.  The  first,  a  control  suljproblem,  defines  an  optimal  re¬ 
turn  function,  as  in  [Ij.  The  second,  a  filtering  subproblem,  defines 

an  optimal  accumulation  function,  T  (^t).  The  dpiamic  programming  problem, 
then,  is  to  find  the  values  of  state  and  parameters,  x,  d,  ^  such  that: 


T  (xu  a,  0)  +  "i'  (.Tt,  a,  ^)  >  T  (xt,  a,  (xt,  a,  0} 


(19) 


4.1  Application  to  Reduced  Order  Robust  Controller 

In  applying  this  technique  to  the  reduced  order  robust  controller,  we  note  that 
there  is  no  direct  parameter  dependence  in  the  performance  index  and  optimal 
return  and  accumulation  functions.  This  is  due  to  the  fact  that,  by  virtue  of 
the  problem  formulation,  tliis  parameter  dej^endence  is  essentially  hidden  in  the 
disturbance  ininit,  u;. 

4.1.1  The  Control  Subproblem 

The  control  subproWem  is  formulated  as  in  (1],  with  a  performance  index  as 
shown  in  (20). 

+  +  l|t'll’v-.)l<^r}  (20) 

Taking  s  0  ,  as  in  [IJ,  and  adding  the  zero  quantity 

i  •^[x^nx]dT  -  ^\x*'nx]\‘ 

to  (20),  tlie  optimal  control  and  state  disturbance  and  a  Riccati  differential 
equation  similar  to  that  obtained  in  (1)  are  determined  by  substituting  and 
completing  squares. 


G 


n{tf)  =  Qt, 

(21) 

uc  =  -R-^B’^Ux 

(22) 

wc  =  ewr'^nx 

(23) 

Substituting  these  relationships  back  into  tlie  performance  index  (20),  an 
expression  for  an  oi)tiinal  return  function  is  obtained.  This  optimal  return 
function  is  then  given  by: 


(24) 

4.1.2  The  Filtering  Subproblem 

Next,  the  filtering  subj^roblein  is  considered.  The  approach  taken  in  solving  this 
portion  of  the  ijroblem  is  essentially  the  same  as  that  outlined  in  (5).  Considering 
only  the  problem  from  initial  time  0  to  current  time  t,  and  making  use  of  the 
measurement  equati<ai  (2),  the  performance  index  for  this  portion  of  the  problem 
can  be  written  as: 


+ 1 +  11*1  -  )lrfr}  (25) 

Adjoining  tlie  dynainic^i  (1)  and  perfect  measurements  (3)  to  the  performance 
index  with  multipliers  A  and  p,  (25)  can  be  rewritten  as: 


"{  f  [X^ {Ax  +  Bu  H-  Tii;  -  x)  +  -  H2x)]dr} 

V  Jo 

Then,  integiating  by  parts  and  solving  for  first  order  necessary  conditions  for 
an  extremum  yield  the  relations: 


x((»)  = 

xo-Po{m  +  Hjii{0) 

(20) 

A  = 

-A'^X  -  -  eQ)x  -  A'^H^ /i  +  HfV-^zi 

(27) 

Wf  = 

-iyr’'(A+//J’/x) 

(28) 

First,  the  initial  conditions  are  considered.  By  combining  the  relations  (20) 
and  22(^1)  =  H2x{i))y  an  exi>ressifin  for  /t(())  is  obtained. 
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/x(0)  =  -  Hixo  +  H2P0XH)))  (29) 

Substituting  this  ex])ression  bade  into  (2C),  we  obtain  the  relationship; 

x(0+)  =  x(()+)  _  P(o+)A(0+)  (30) 

i(0+)  =  xo  +  PoHj(H2PoHl)-\z2{Q)-HiXo)  (31) 

P(0+)  =  Po-PoHUH2PoHJ)-^H2Po  (32) 


This  represents  the  state,  costate,  state  estimate,  and  Riccati  solution  at  the 
time  when  the  first  measurement,  22(0),  is  received.  An  important  observation 
is  that 


//2t(0+)  =  52(9) 

//2P(«+)  =  0  P(0+)Hj  =  0  H2P(()+)HJ'  =  0 

That  is  to  say,  once  the  first  measurement  is  obtainerl,  that  portion  of  the  state 
contained  in  the  irerfect  measurements,  22,  is  known  perfectly.  Also,  the  Riccati 
solution  P(-)  bmnnes  singular  once  measurements  are  taken. 

To  obtain  an  expression  for  /i  for  time  t  >  0,  we  can  combine  (1)  and  (28) 
with  the  constraint  Z2{  )  =  which  >1elds  the  expression: 


//(■)  =  -(H2TWr^H^)-\i2{  )  -  H2[A:r.{.)  +  B«(.)  -  rHT^A(  )|}  (33) 

This  relationship,  along  with  (28)  c<in  then  be  substituted  bac^k  into  (1)  and 
(27)  to  give  a  system  of  differential  equations  for  x  and  A. 


+ 


A  -rifT^  1  r  X 1 

-{H'^R-^H-eQ)  J[aJ 

]“■*■[  -^R-'z  ] 


where  the  quantities  alcove  are  reprasenterl  by: 


(34) 


R2  =  H2TWT‘^HJ 

Ar^\I-GH2]A 


Hi  =  H2A 


I-GH2\B 
’  V  0  ] 
0  Ri  I 


G  = 

r\vr^  =  [i-GH2]r\vr^ 


Next,  as  in  [4],  differential  equations  for  the  propagation  of  the  state  estimate 
i(-)  and  Riccati  solution  P(  )  are  obtained  by  differentiating  the  relationship 
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x(0=iW-P(/.)A(/.) 


(35) 


This  results  in  the  differential  equations: 

i  =  {A  +  0PQ)x  +  (B-PHJ'R^^H2B)u 

+PR'^R- ^{z-  Hix)-^Gz2  (36) 


P  =  AP  +  PA'^ -P(H'^R-^H -0Q)P  +  rWr'^ 

■P(<l)  =  Po  (37) 

It  should  he  noted  that  the  differentia!  equation  which  governs  the  dynamics 
of  the  state  estimate,  x,  contains  the  derivative  of  the  perfect  measurements,  Z2. 
For  the  sake  of  implementation,  this  is  iK»t  desirable,  but  can  easily  be  remedied 
by  defining  a  transformed  state  estimate  which  contains  only  the  measurement 
itself,  Z2  (5j.  The  estimator  con  then  be  implemented  by  using  the  transforma¬ 
tion 

x  =  x-  {pH-f^fq  ‘  -H  g)  Z2  (38) 

The  next  step  is  to  detennine  the  optimal  accumulation  function.  To  do 
this,  we  first  substitute  the  relation.ships  (28)  and  (33)  into  (25)  and  add  the 
zero  quantity 

Then,  the  performance  index,  (2r>)  reduces  to 

jfpA  =  ~><^p><+yy\rr\\%+Mi 

“^11^1  “  ~  —  H2iAx+  S-«)||^_i]dT  (39) 

Next,  we  would  like  to  exjjress  this  i>erfonnance  index  in  terms  of  the  error 
between  actual  and  estimaterl  state,  c{  )  =  x(  )  -  x(  ).  Care  must  be  taken, 
since  P(  )  is  singular  for  all  1  >  0.  Fii-st,  we  define  a  transfomiwl  error,  e(  ): 
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where  Ho  is  defined  to  span  the  portion  of  the  state  space  not  contained  in  the 
perfect  measurements  22(*)-  Also,  a  transformed  oostate  A(')  is  defined  such 
that 


Then,  using  tliese  relationsliips,  we  can  write 

A^PA  =  el{HoPH^r% 

=  (x  -  xfH^iHoPH^r^Hoix  ^  x)  (42) 

Substituting  back  into  the  jwfonnance  index,  we  then  obtain  tlie  desired  ex¬ 
pression  for  the  o])tiinal  accumulation  function: 


T(.t,)  =  y{u,  zuh)  - 


(43) 


where 


~\i-2  -  i/2(Ai  +  -  H2{Ax  +  Bu)l 

</(-.-.  )t=0  =  11^2 (44) 

4.1.3  The  Connection  Condition 

Having  determined  exj^ressions  for  the  optimal  return  and  optimal  accumulation 
functions,  we  then  nmst  determine  the  connec’.tion  conditions  which  bring  to¬ 
gether  the  two  solutions.  The  cfumection  cxmdition,  then,  is  obtained  similarly 
to  the  method  presented  in  [1],  from  the  constrained  relationship 


max 


{xut)  +  T(xe,/)  -h  f  -  H^xM 


which  is  solved  by  considering 


=  n-r  -  [HoPH^r^Ho{x  -x)- 


(45) 


(40) 
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The  desired  relationship  between  the  optimal  state  and  state  estimate,  then,  is 
determined  by  combining  the  alx)ve  with  the  constraint  relation  Z2{i)  =  ^23^(0 
and  solving  the  matrix  equation: 

■  {hj  ^0  -  mt))  I  j  ®  J  =  1^  {HoPm^r'  j 

Combining  this  relationship  with  the  optimal  control  (22)  and  a  transformed 
version  of  the  reduced  order  optimal  state  estimate  (3G),  we  can  then  construct 
a  reduced  order  compensator  for  the  system* 

Furthermore,  by  examining  the  second  variation  in  the  manifold  of  the  con¬ 
straint  Z2  =  H2^%  we  can  obtain  an  expression  for  a  spectral  radius  condition 
for  tlie  reduced  order  compensator.  This  is  then  given  by: 

>0  (48) 

where 

P  =  {HoP{t)H^y' Ho 

=  I  -  Hj  Ha 

Note,  by  the  definition  of  Ho,  we  have  HoH^  =  H2HQ  =  0,  so  we  can  rewrite 
the  above  as 


p  -  >  0  (40) 

4.2  Application  to  Adaptive  Controller  with  Some  State 
Coefficients  Estimated 

In  applying  the  d>maniic  ]>rograiinning  technique  to  the  augmented  system  (12), 
we  first  rewrite  the  perfonnance  index  to  reflect  the  augmented  state  vector,  as 
in  (50). 

j  =  l  {lie  (</)iiJ,  - 1  ||a«)  -  +  Ml  - 1  (iiHi?v-.  +  mU)]  (50) 

where,  to  reflect  the  extended  dynamic  system,  Q  and  0/  ?^re  defined  as 


Qf 

0 

0  “ 

Q  = 

“  Q 

0 

0  ' 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

We  can  then  break  the  problem  into  a  separate  control  and  filtering  problem, 
and  determine  the  connec.tion  condition  which  joins  the  two. 
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4.2.1  The  Control  Subproblem 

The  control  subproblem  is  formulated  as  in  (1],  with  a  performance  index  as 
shown  in  (51). 

Jc  {tM  =  5  |l|x [||x||*  +  Ml  - 1  )]  (51) 

Taking  v*/  =  0  ,  as  in  (1),  and  adding  the  zero  quantity 

I  ^  ^  ^  [x’’  (r)  n  (o,  0,  t)  x  (r )]  tir  -  |  (•)  11  (a,  0,  •)  x  (-)] 

to  (51),  the  optimal  control  and  state  disturbance  and  a  Riccati  differential 
equation  similar  to  that  obtained  in  [1]  are  detennined  by  sul>stituting  and 
completing  scjuares. 


-n  (a,  0,r)^  (a,  /?.  r)  +  H  (a,  ^3,  t)  X  +  Q  - 

n (a,  0,  T)  -  eniT^)  n  (a, 0,r)  (52) 

t  <T  <ij 

n  («,  0,  tj)  =  Qj 

n(t)  =  -fl-’B^n(a,(S,t)x(t)  (53) 

«;  (#)  =  OHT^n  (a,  0,  t)  x  (t)  (54 ) 

Substituting  these  relationships  back  into  the  performance  index  (51),  an 
expression  for  an  optimal  return  function  is  obtained.  Tliis  optimal  reUini 
function  is  then  given  by: 


^{xt,a,/3,)-^xln  (a, /?,  t) xt  (55) 

4.2.2  The  Filtering  Subproblem 

Next,  the  filtering  subproblem  is  considere^l.  Tlje  approadi  taken  in  solving  this 
portion  of  the  i^roblem  is  essentially  the  same  as  that  outlined  in  (5),  Considering 
only  the  problem  from  initial  time  0  to  current  time  t,  and  making  use  of  the 
measurement  ec|uation  (7),  the  j>erformanc^e  index  for  this  portion  of  the  problem 
can  be  written  as: 
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Jj  10. *1  =  \  {-i  ||e(0)  -  [llello  +  II«IIr  -  \  (ll«'ll?.'-  +  II^I  -  dr'^  (50) 

Adjoining  the  dynamic  constraints  to  the  performance  index  with  multipliers  A 
and  //,  Jf  can  then  be  written  as 

Jf  (0,  t]  =  J/  [0,  t]  —  -  +  B^u  +  ^ j  {z2  “  J^20] 

Then,  integi'ating  the  fi  terms  by  parts  and  solving  for  first  order  necessary 
conditions  for  an  extremum  yield  the  relations: 


C  (0)  =  eo  -  Po  (A  (0)  +  Hhi  («))  (57) 

/t  ((>)  =  -  (H-iPoHl) ~ *  [^2  (0)  -  H2io  +  HaPoA  (())]  (58) 

A  =  -AjX  -  {Hj'V-'Hi  -0Q)i-  AjHjii  -  (59) 

«;  =  (A  +  Kj ft)  (CO) 


Combining  the  constraint  (0)  =  (0)  with  the  relationship  obtained  for 

/t{()),  we  get  an  exinession  for  the  initial  conditions  upon  rec*eiving  the  first 
niejisiirenient,  z^  (0) 


^  (0+)  =  i  (0+)  -  P  (04-)  A  (0+)  ((il) 

where 

i  (0+)  =  io  +  {HiPoHjy'  {z2  (0)  -  H2io)  (02) 

P  (0+)  =  Po-  PoHj  {HiPoHjy^  H2P0  (03) 

An  iiniKutant  result  of  these  relationshijw  is  that  upon  receiving  the  first  “per¬ 
fect”  measurement,  the  estimate  of  the  ]>erfectly  measured  states  is  then  simply 
22,  and  the  matrix  P  Ijecomes  singular,  with  null  space  corresimnding  to  the 
portion  f)f  the  state  space  which  is  measured  perfectly.  That  is, 
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H2C(«+)=-2(0) 

H2P  (0+)  =  0  P  (0+)  =  0  H2P  (0+)  H^  =  0 

Next,  to  obtain  an  expression  for  /*  for  time  f  >  0,  we  consider 


Z2  =  Hii  =  H2  [>l4^  +  B^U  -  r^wrj  (A  +  Hjfi)]  (C4) 

Then, 


H(.)  =  -  {H2TiWrlHj)  ‘  [i  -  Pa  (v4<e  +  B«w  -  r^lVF^A)]  (C5) 
Tliis  then  re,snlts  in  a  system  of  riifferential  equations  given  by 


A 


-OQ) 


-r^Wrl 


+ 


HfV-^  BJR^^ 


m 


with  tlie  matrices  alxwe  defined  jus 


Pa  =  Par^HT^  HJ  Hi  =  HiA^ 

Ai  =  \I-GHi]Af,  P4  =  (/-GPalB4 


G’  =  r4ivrfpJ’Pj* 
r^H/re^’  =  \i-  GPai 


Then,  differentiating  the  relationship 


(07) 

we  obtain  difTerential  relations  for  an  estimator  and  Riccati  ecjuation  given  by 

I  =  {Ai  +  0PQ)i+{Bi-PHlR^^H2Bi)u 

+PhTV- ‘  (z,  -  PiC)  +  PHJ PJ  *  (ia  -  PaC)  +  Gz2  (C8) 


P  =  A^P  +  PAl-P{H'^R-^H-eQ)P  +  riWrf  (f.9) 

P(())  =  P(0+) 
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As  in  the  r4tse  of  tlie  robast  controller  design,  it  is  desirable  to  introduce  a 
transformation  which  eliminates  the  measurement  derivative,  ia  from  tlie  esti¬ 
mator  equation.  Tins  transformation  is  given  for  this  j^tem  by 

i=i-{pnjii^^+G)z2  •  •  (70) 

The  next  step  is  to  detennine  the  optimal  accumulation  function.  To  do 
this,  we  first  suljstitiite  the  relationsln|>s  obtained  alcove  into  Jj  and  add  the 
zero  quantity: 


Then,  we  can  rewTite  Jj  as 


2  lo  [INIq  ^  "  I  h  ~  ~  IL 

Next,  we  define  tlie  error  between  augmented  state  and  estimate  as  e  =  f  -f, 
and  define 


’  to  " 

■  Ho  ■ 

C‘2 

i/2 

Cq 

Ha 

.  - 

where 

//«  =  [  0  0  /fcx/t  0  ]  //^  =  [  0  0  0  ] 

Also,  the  multiplier  A  ctaii  be  written  as 


■  Ho  ■ 

T 

■  Ho  ■ 

T 

'  h  ■ 

Hi 

X  — 

Hi 

A2 

Ha 

Ha 

Xa 

1^0  1 

.  V  . 

Combining  tenns,  we  can  then  write 

X'^PX  =  ej{H^PHTy'e^ 


(72) 


(73) 


(74) 


ir, 


s 


where  =  //^e,  and 


’  ^  * 

■  Ho  ' 

Ca 

//|  = 

.H0\ 

The  optimal  accumulation  function,  then,  can  be  written  as 

T(Cf)  =i/(tt,zi,i2)  -  (75) 


where  S  {t)  is  defined  as 


(HfPHiy'Hi 


and 

ilin^zuk)  =  i\^Qi]-±{zr-HdfV-\zr-Hii) 

+  Biu)fR^^\z2  -  H2(A4  +  B^i)] 

4.2.3  The  Connection  Condition 

Having  determined  exinessions  for  the  oj^timal  return  and  accumulation  func- 
ti€)ns,  we  must  then  determine  the  connection  conditions  which  join  the  two 
solutions.  As  in  [1],  we  fii*st  partition  S(t)  as 


5{f.)  = 


sL 

L  ^^0 


Sxfl 


Then,  as  in  tlie  case  of  the  reduced  order  robust  compensator,  we  adjoin  the 
constraint  Z2  =  H-zC*  Perfonning  the  optimization  with  respect  to  the  state 
yields: 


=  n,  {a,0):r,  -  i  |^Sx  (a:,  -  x, )  +  5*a  (a-o)  +  5j:p 


l(i 


This  relationship,  combined  with  the  ctonstraint  of  the  perfect  measurements, 
yields  the  state  connection  condition. 

Next,  solving  for  the  state  coefficient  parameter  connection  condition  yields 

(X|  -  x,f  +  (a  -  5a  +  -  py  Sl^  =  |a:r  (78) 

The  control  coefficient  parameter  connection  condition  is  given  by 

(a:,  -  XrfS^p  +  (a-  &f  +  (p  -  pf  (79) 

The  connection  conditions  can  then  l>e  used  to  provide  the  link  between  tlie 
estimation  problem  and  tlie  control  inoblem. 

5  Conclusions 

The  results  presented  in  this  i>ai>er  extend  the  existing  results  in  robust  (2,  4] 
and  adaptive  [1]  control  based  on  disturbance  continuation  to  a  class  of  problems 
where  a  combination  of  i>erfect  and  iin]>erfect  state  information  is  available.  In 
the  C4ise  of  the  robiLst  controller,  this  allows  for  the  reduction  of  the  required 
order  of  the  compensator  in  such  systems.  This  may  be  of  practical  impor¬ 
tance  in  applications  where  some  measurements  are  known  nearly  exactly,  and 
compensator  size  is  of  cxmcern. 

In  the  adaptive  cjise,  the  ap]>roach  used  in  this  paper  allows  for  the  esti¬ 
mation  of  parameters  multij^lying  i>erf«!tly  measured  states,  whicJi  may  help  to 
increase  jKjrfonnaiK^e  in  some  uncertain  systems  with  some  measurements  whidi 
are  known  exactly. 
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FOR  THE  F-18  HARV 
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Parameter  Robust  Game  Theoretic  Synthesis  for  the  F-18  HARV 

Charles  H.  Dillon*  and  Jason  L.  Speyert 
Ihiiveisity  of  California,  Los  Angeles 


Abstract 

A  parameter-robust  game  theoretic 
compensator  has  been  designed  for  the  F-18  High 
Angle-of-attack  Research  Vehicle  (HARV). 
Modelling  uncertainties  due  to  parameter  variations  in 
the  aircraft  dynamics  over  varying  flight  conditions 
are  included  in  the  design  process  by  u^g  a  internal 
feedback  loop  OH.)  deCmnposition  and  incorporating 
die  resulting  state  space  into  a  ^turbance  attenuadon 
problem.  The  compensator  is  designed  to  trade  inlot 
inputs  in  angle-of-attack,  sideslip,  and  stabili^-axis 
roll  rate  through  stick  and  rudder  pedal  commands. 
Zero  steady  state  oior  in  the  presence  of  step  inputs 
is  achieved  by  using  a  system  of  error  coordinates 
udiich  also  allows  for  thrust  vector  commands  to  fade 
to  zero  in  steady  state.  Linear  and  nonlinear 
emulation  results  are  presented. 

1.  Introduction 

The  effort  to  date  in  the  development  of  a 
parameter-robust  controller  for  the  F-18  high  angle- 
of-attack  research  vehicle  (HARV)  using  game 
theoretic  amtroUers  [1]  has  Concentrated  mainly  on 
compensator  design  and  evaluation.  The  design 
objective  was  to  create  a  compensator  capable  of 
trademg  state  command  inputs  in  the  presence  of 
uncertain  dynamics  at  high  anglerof-attack. 
Additionally,  physical  limitations  of  the  thrust 
vectoring  vanes  dictated  that  the  design  be  such  that 
thrust  vectoring  be  used  mainly  for  enhancement  of 
transient  response,  with  aaodynamic  ccmtrols  being 
used  in  steady  state.  The  approach  taken  in  designing 
the  control  system  was  a  linear  design  teduuque  based 
on  game  disturbance  attenuation  [1]  with  parameter 
uncertainty  included  in  the  design  process  as  a 
fictitious  disturbance  via  an  internal  feedbadc  loop 
(IFL)  decomposition  [2].  As  with  other  linear 
controllers,  this  tedinique  utilizes  a  linearized  model 
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of  the  aircraft  about  a  particular  flight  cemdidon.  The 
game  theoretic  synthesis  essentially  produces  a 
controller  which  r^uces  the  senfidvity  of  the  closed 
loop  system  to  disturbances.  By  inclutdng  parameter 
uncertainty  as  a  fieddous  disturbance,  the  sensidvi^ 
of  the  closed  loop  system  to  parameter  variadons  is 
reduced,  and  this  decomposidon  helps  to  extend  the 
region  in  which  the  resuldng  linear  control  gains  can 
be  used.  Iberefore,  by  extending  the  usable  region 
about  each  design  point,  fewer  design  points  are 
required,  thus  reducing  the  number  of  gains  to  be 
Stored  in  the  final  implementadon  of  the  controller. 

Tbe  main  analysis  tool  used  in  generating 
linearized  models,  obtaining  estimates  of  the 
parameter  uncertaindes,  and  evaluating  controller 
performance  has  been  the  Dryden  F-18  HARV 
nonlinear  batdi  dmuladon.  Ibis  simiiladon  protddes 
a  detailed nonlinear  model  of  the  rurcraft,  including  an 
aerodynamic  database  containing  data  for  "clean" 
(angle-of-attack  less  than  40  degrees),  high  alpha 
(angle-of-attadc  between  40  and  %  degrees),  power 
approach  and  takeoff  flight  regimes.  The  simuladon 
is  currently  hosted  on  a  SUN  workstadon  at  UCIA. 
Capability  for  using  interchangeable  control  laws, 
through  a  simuladon  control  law  interface  provided 
by  NASA  Dryden  exists  within  the  simuladon  and 
has  been  used  in  testing  a  FORTRAN 
irnplmnentadon  of  the  linear  controller  with  the  full 
nonlinear  aircraft  dynanucs. 

2.  Mndglling  Assumptions 

The  full  nonlinear  equadons  of  modon  for 
the  F-18  HARV  are  given  below  (3,4].  The 
moments,  denoted  L,  M,  and  N,  and  the  forces, 
denoted  Xw,  Ttw,  and  Z^,  where  the  subscript  w 
indicates  a  wind  axis  reference  frame,  have 
components  due  to  gra\itadon,  aerodynamics,  and 
engine  thrust  These  forces  and  moments  ate  strongly 
dependent  on  the  values  of  state  and  control 
deflections.  The  vehicle  mass  properties  are  reflected 
in  the  inertia  tensor,  /,  and  velucle  mass,  m.  Aircraft 
velodty  is  given  by  Vand  body  angular  rates  by  p,  q, 
andr.  The  matrix  T/p  is  a  coordinate  transformation 
from  body  to  an  inertial  reference  frame,  which  is  a 
function  of  the  Euler  angles  0,  and  vt  The  matrix 
Tsyf  is  a  coordinate  transformation  from  wind  to 
body  axes,  which  is  a  function  of  angle-of-attack  and 
sideslip.  The  rates  ps  and  indicate  stability  axis 
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roll  and  yaw  rates,  respectively.  Tlie  dynamics  are 
given  as: 
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dt 
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where  the  A  and  B  matrices  contain  the  linearized 
force  and  moment  coefficients  at  the  ^en  flight 
condition. 

A  vital  part  of  the  controller  design  was  in 
determination  of  the  model  uncertain^  for  use  in  the 
synthesis  procedure.  To  accomplish  this,  several  trim 
conditions  about  the  nominal  were  selected  and  linear 
models  were  geaerated  at  these  points,  linear  model 
coefficients  wm  then  compared  at  the  vaiybg  flight 
conditions  to  determine  whidi  coefficients  iqHesenled 
the  greatest  sources  of  nncertain^r.  Upon 
examination  of  these  variations,  it  was  found  that 
many  of  tiie  coefficients  varied  in  magnitude  over  the 
ranges  conridered,  but  some  varied  in  rign  as  well. 
The  state  coefficients  that  seemed  to  exhibit  the 
greatest  amount  of  uncertainty  were  Lp,  Ma.  aadNp. 
The  control  coefficients  whidi  varied_the  most 
significanUy  were  YsAiLr^SHTr^^RDRy^^^ 
N  shTt'  ^  sample  of  the  variations  in  these 
coefficients  is  shown  in  Rgs.  1-6. 


3.  Compensator  Design  and 

Implgme.nta.!Ho>i 

3.1  Design  of  State  Space  for 
Compensator  Design 


For  the  controller  deagn,  a  linearized  model 
was  extracted  from  the  nonlinear  vehicle  dynamics  at 
a  particular  flight  condititm.  A  5*state  system  was 
chosen  to  represent  the  dynamics  of  interest  in 
controller  design  which  included  angle-of-attadc  a„ 
rideslip  p,  and  roll,  pitch  and  yaw  body  angular  rates 
(p,  q,  r).  Control  surfaces,  a  total  of  8,  included 
leading  and  trailing  edge  flaps  (St  vf.  Stef\  nidder 
(Srdr),  differential  and  collective  horizontal  tail 
iSffr),  differential  aileron  and  pitch  and  yaw 

thmt  vectoring.  The  lineaj^ed  modd  then  had  the 
form  as  shown  below,  lliis  model  was  generated 
using  the  linearization  routine  in  the  nonlinear  batch 
simulation  at  a  given  trim  condition.  The  condition 
selected  for  controller  design  was  at  an  angle-of-attack 
of  SO  degrees  and  an  altitude  of  2S000  ft  The  states 
of  the  linearized  model,  then,  represent  perturbations 
of  the  nonlinear  model  states  about  their  nominal 
trim  values.  Actuator  dynamics  were  not  included  in 
the  riate  space  used  for  design,  the  linear  dynamical 
—  system  is  then  represented  by:  ~ 

X  =  Ax  +  Bu 
x-[a.p.p,q,rf 
SiEFf,  Stef^  Stvc^  Pul„  Sht^  &Z)R,, 


The  goal  in  designing  the  ccunpensator  for 
the  linearized  dynamic  system  was  to  track  state 
command  inputs  such  that  steady  state  enors  due  to 
st^  inputs  would  go  to  zero.  This  was  aooaaq>lished 
by  uring  a  system  of  error  coordinates,  ei  =  x-Xc, 
where  Xe  is  tiie  input  vector  step  command.  By 
differentiating  the  error  coordinates,  the  resulting  state 
space  is  a  linear  system  haring  the  derivative  of  the 
control  surface  deflections  as  a  controlling  input, 
which  decays  to  zero  in  steady  state  in  the  presence  of 
step  commands  [5,6].  Also,  it  was  desired  that  the 
Arust  vector  control  conunands  be  thiveo  to  zero  in 
steady  state  to  avoid  damaging  the  thrust  vectoring 
control  vanes.  This  was  accomplished  l^  defining  as 
a  state  a  subset  of  the  controls,  vj-  where  Bw 

is  defined  sudi  that  vj  =  [Srvcpt  Sjvcyl^^  The 
resulting  linear  state  equations  woe  then  given  by: 
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To  model  parameter  variations  in  the 
linearized  state  space,  an  internal  feedbadc  loop  QOFL) 
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decompositioD  was  used  [2,7].  Ibis  decompoadon  is 
iquesented  by  the  system : 

Xtn{Ao  + AA)Xe  +  {Bo*  ^B)u  ^4) 
AAr^D  LJie)  E  AB  =  F  LiJ^e)  G 

The  matrices  I^e)  and  Lj/e)  contain  fimcdtHis  which 
describe  the  form  of  the  parameter  variations,  uith  e 
in  this  case  representing  uncertainty  in  the  linearized 
state  and  control  coefiSdents..  By  decomposing  the 
parameter  variations  as  such,  the  system  can  be 
rewritten  in  a  form  where  the  parameter  variations  act 
as  ficritious  disturbances  to  the  nominal  system. 
This  resulting  system  is  dien  written  as: 


The  disturbance  attenuation  problem  can 
then  be  converted  to  a  differendal  game  problem  with 
a  quadratic  performance  index  given  by: 

Xu,w,t^=  [  + yfyi  -  (S) 

Jo 

The  solution  of  this  problem,  as  presented  in  [1]  is 
obtained  by  allowing  final  Ume  to  extend  to  infinity, 
and  then  solving  the  resulting  algebraic  Riccati 
equation  (ARE): 


it  =  Aox*  +  Bqu  +  I\v 


r=[z)  f] 


where  F  is  a  new  disturbance  input  matrix  which 
includes  the  ficfidous  disturbances  representing 
parameter  uncertainties,  y/  is  an  ou^ut  vector 
associated  with  the  parameter  uncntainty  and  w  is 
conrideted  as  die  disturbance  vector. 


A^+  JIAq  -  J7|  BoR-^bI—^FT^  j  JT+  g  =  0 

Q  =  pC^C*E'^E  (9) 

R  =  pCfCi  +  G^G 

This  algebraic  Riccati  equation,  however, 
has  a  sign  indefinite  quadratic  term.  The  parameter  y 
must  be  adjusted  until  a  solution  exists.  As  gamma 
extends  to  infinity,  this  becomes  equivalent  to  the 
standard  linear  quadratic  regulator  problem.  The 
resulting  controller  obtained  by  solving  the  ARE, 
then,  is  a  linear  combination  of  the  states  of  the 
form: 


Once  the  system  has  been  written  in  the 
above  form,  a  controller  can  be  designed  which 
reduces  the  sensitivity  of  the  closed  loop  system  to 
parameter  variations,  acting  as  disturbances  to  the 
nominal  linearized  system.  This  is  done  by  first 
constructing  a  disturbance  attenuation  function,  which 
is  essentially  the  ratio  of  desired  performance  outputs 
to  disturbance  inputs.  For  all  admissible 
disturbances,  a  controller  must  then  be  found  which 
bounds  this  fimction: 


sup  {  ^^y  *y\y^dt /  [  w^wdt  <y^ 

»v6la|Jo  Jo 


The  vector  y  represents  a  linear  combination  of  states^ 
and  controls  to  be  weighted  quadratically  to  attain  a 
desired  level  of  perfonnance  and  p  is  a  scalar  which  is 
used  to  adjust  the  relative  weighting  between 
perfonnance  and  sensitiri^.  The  performance  ouq>ut 
y  is  rquesented  by: 


,  u  =  -R-^Blnx  (10) 

For  the  error  coordinate  system,  this  results 
in  the  derivative  of  the  control  being  a  function  of  the 
state  errors,  their  derivatives,  and  the  thrust  vector 
comiitands.  The  control,  then,  results  in  a  linear 
ccxnbination  of  the  error,  the  integral  of  the  errw,  and 
the  integral  of  the  thrust  vector  conunands. 

«  =  -[  Kei  Ke2  Ky^  ]  ||  j  Vi^j  (11) 


The  resulting  compensator  is  essentially  a 
proportional  plus  integral  system.  A  blodc  diagram 
of  the  implementation  of  the  above  controller  is 
shown  in  Figure  7.  State  command  inputs  are 
obtained  as  a  function  of  pQot  longitudinal  and  lateral 
stick  and  rudder  pedal  inputs.  The  baseline 
implementation  commands  angle-of-attack  as  a 
function  of  longitudinal  stick,  stability  axis  roll  rate 
as  a  function  of  longitudinal  stick,  and  sideslip  as  a 
function  of  rudder  pedal.  This  combination  was 
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selected  to  be  somewhat  cotisistent  with  the  baseline 
Research  Flight  Control  System  [8].  It  should  be 
noted,  however,  that  any  ocunbination  of  the  5  slates 
may  te  commanded  by  combinations  of  diese  i^ot 
inputs  if  desired.  Fm  example,  a  {ntdi  rate  oonunand 
system  would  be  just  as  easily  implemented,  as  the 
baseline  implementation  simply  uses  zero  as  the 
commanded  pitdi  rate.  A  first  order  prefilter  was 
added  with  natural  frequency  of  5  rad/s  to  effectively 
limit  the  bandwidth  of  ccanmand  inputs  so  as  to  avoid 
acmatoriate  saturation. 

The  selection  of  the  state  and  control 
weightings  C  and  Cj  was  made  such  that  reasonably 
fast  step  response  to  command  inputs  could  be 
achieved  without  causing  actuator  saturation.  A 
baseline  controller  was  first  designed  without 
including  uncertainty  in  order  to  determine  a 
reasonable  set  of  state  and  control  weightings  to  be 
used.  Once  th^  were  determined,  the  IFL 
decmnposition  was  used  to  include  uncertainty  effects 
in  the  controller  design.  The  parameters  p  and  y  were 
adjusted  accordingly  to  produce  a  controller  which 
bounds  the  associated  disturbance  attenuatitm  function 
and  provides  a  reasonably  good  balance  between 
system  perfcamance  and  parameter  senativity. 

_ Simulation  Results 

In  order  to  evaluate  the  controller  design, 
^ulations  were  performed  using  both  the  linearized 
model  and  the  lull  nonlinear  batch  .simulation  model. 
To  generate  nonlinear  simulation  results,  the 
controller  was  implemented  as  a  FORTRAN 
subroutine  and  connected  to  the  batch  ^ulation  via 
the  control  law  interface  provided  by  NASA.  Linear 
simulation  results  were  generated  using  linearized 
models  in  MATLA6.  A  nominal  trim  flight 
condititm  at  angle-of-attadc  of  SO  degrees  and  altitude 
of 25000  feet  was  sdected  to  perform  the  evaluations. 
Step  inputs  in  angle  of  attack  and  doublets  in 
stability  axis  toll  rate  were  performed  in  both  the 
linear  and  nonlinear  simulations.  Comparisons 
between  control  designs  syntheazed  vitii  and  without 
the  presence  of  parameter  uncertainty  were  made  in 
both  the  linear  and  nonlinear  case.  Gains  generated 
with  the  effects  of  parametu'  uncertainty  used  a  value 
of  1000  for  the  parameter  y.  Results  from  the 
nonlinear  simulation  were  then  compared  to  linear 
results  to  verify  the  effectiveness  of  the  controller  at 
flight  conditions  varying  from  the  nominal  trim 
condition.  _ 

4.1  Linear  Simulation  Results 


and  control  performance  weightings  were  adjusted  in 
the  derign  process  to  provide  adequate  time  response 
to  step  inputs  in  the  cortunanded  states,  without 
requiring  overly  large  commands  from  any  of  the 
controls.  As  the  effect  of  parameter  uncertainty  was 
added  to  the  design,  comparistms  could  be  made  to 
show  tile  resulting  effect  on  time  response. 

Figures  8-9  show  time  responses  of  the 
linear  system  usmg  gains  generated  vtith  and  without 
parameter  uncertainty.  Linear  step  responses  in 
angle-of-attack  and  doublet  responses  in  stability  axis 
toll  rate  are  presented,  linear  responses  to 
hmgitudinal  step  inputs  are  rnrtually  identical  with 
and  without  uncertainty  induded  in  the  derign,  vbile 
there  is  a  slight  difference  in  lateral  response.  This  is 
mainly  due  to  the  fact  that  most  of  the  uncertainty 
accounted  for  in  the  design  process  was  in  the  lateral 
coeffidents. 

4.2  Nonlinear  Simulation  Results 

Responses  were  generated  in  the  nonlinear 
batch  simulation  starting  from  the  nominal  trim 
condition  of  50  degrees  angle  at  attadc  at  25000  feet 
altitude.  Stq>  responses  in  angle-of-attack  were  fairiy 
consistent  near  the  nominal  point,  but  had  some 
difficulty  adiieving  the  higher  angle-of-attack  flight 
conditions  due  to  actuator  saturations.  This  is 
illustrated  in  Figure  10.  Lateral  responses,  shown  in 
Figures  11-12,  showed  only  small  differences  between 
Ihededgn  nrim  and  without  tiie  effects  of  uncerhunty. 
The  main  difference  between  the  two  can  be  seen  by 
examining  the  actuator  responses  shown  in  Figures 
13-16.  Tbe  parameter  robust  oontrollo'  tends  to  use 
thd  tiirust  vectoring,  whose  effect  is  known  with 
greater  certainty,  than  the  aerodynamic  controls, 
which  are  less  well-known. 

L _ Conclusions 

A  parameter  robust  controller  based  on  game 
theoretic  synthesis  has  been  designed  for  the  F-18 
HARV  and  implemented  in  nonlinear  batch 
Emulation.  The  goal  of  this  design  was  to  be  able  to 
track  pilot  command  inputs  in  the  presence  of 
parameter  uncertainties.  A  linear  controller  was 
designed  and  game  theoretic  syntheris  used  to  extend 
the  valid  range  of  the  linear  ccmtroller.  The  use  of 
error  coordinates  in  the  controller  design  provided  for 
a  compenrator  which  achieves  zero  tracing  error  to 
step  command  inputs  in  steady  state.  Also,  tbe 
cranpensator  design  allows  for  any  combination  of 
angle-of-attack,  sideslip,  roll,  pitch  and  yaw  body 
rates  to  be  conunanded  with  zero  steady  state  error. 


Linear  simulation  results  were  generated  to 
determine  tbe  response  of  the  controller  in  the  linear 
region  about  the  nonunal  trim  operating  point.  State 
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From  simulation  results,  it  was  seen  that 
this  controller  produced  some  improvement  in  the 
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Disturbance  Attenuation  Approach  to  Adaptive 
Control  Applied  to  Longitudinal  Flight  Control  of 

the  F-18  HARV 

C.  H.  Dillon  and  J.  L.  Speyer 
University  of  California,  Los  Angeles 

Abstract 

The  disturbance  attenuation  apj^roach  to  controlling  systems  subject 
to  external  disturbances  and  uncertain  coefficients  has  recently  been  un¬ 
der  investigation.  In  this  api>roach,  a  ratio  of  performance  outputs  to 
disturbance  inputs  is  formed,  and  a  controller  is  found  which  maintains 
this  ratio  below  a  certain  bound.  Tliis  disturbance  attenuation  problem 
is  solved  by  converting  the  disturbance  attenuation  function  to  a  perfor¬ 
mance  index  of  a  dynamic  game  problem.  This  game  problem  is  then 
solved  iLsing  dynamic  programming.  Tlie  resulting  compensator  structure 
is  adaptive  in  that  estimates  of  uncertain  coefficients  are  calculated  on 
line,  and  robast  in  tliat  the  control  is  a  function  of  the  worst  case  state 
and  parameter  values.  This  i>aj>er  demonstrates  the  apjdication  of  such 
a  compensator  to  the  problem  of  longitudinal  flight  control  with  imcer- 
tain  aerodynamic  control  moment  coefficient  and  well  known  thrust  vector 
control.  Step  responses  in  angle  of  attack  are  presented  which  illustrate 
the  effect! vencvss  of  the  robiLst  adaptive  compensator. 

1  Introduction 

Over  the  i>ast  several  years,  techniques  have  been  developed  for  controlling 
uncertain  linear  systems  sulqect  to  external  disturbances  by  considering  a  dis¬ 
turbance  attenuation  problem.  In  this  approach,  a  measure  which  is  essentially 
the  ratio  of  norms  of  performance  outjiuts  to  disturbance  inputs  is  created,  and 
a  robust  comjjensator  is  sought  which  bounds  this  ratio  below  some  limit.  In 
[1,  2],  this  i^roblem  was  approached  by  converting  this  disturbance  attenuation 
function  to  a  performance  index  and  then  using  a  game  theoretic  approach  to 
find  the  minimizing  control  for  the  worst  case  maximizing  disturbance.  This 
approach  extended  the  results  of  analysis  to  include  not  only  time  invariant 
systems  on  infinite  intervals,  but  time-varying  systems  on  finite  intervals  as  well. 

In  the  work  presented  in  [3],  this  disturbance  attenuation  approach  was 
applied  to  a  chuss  of  pi  oblems  in  which  uncertainty  exists  in  one  or  more  param¬ 
eters  in  the  system  control  coefficient  matrix.  Using  a  dynamic  programming 
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approach  suggested  in  [1],  the  problem  was  split  into  two  parts.  Optimizing 
from  current  to  final  time  yielded  a  controller  as  a  function  of  the  states  and 
parameters  and  an  oj>tiinal  return  function  which  represents  the  cost  to  go  from 
the  current  time  to  the  final  time.  Optimizing  from  initial  to  current  time 
yielded  an  estimator  structure  which  provides  estimates  of  the  current  state 
and  parameter  values  based  on  past  measurements  up  to  the  current  time  as 
well  as  an  optimal  accumulation  function  which  represents  the  cost  accumulated 
up  to  the  current  time.  An  algebraic  “connection  condition”  was  then  deter¬ 
mined  by  maximizing  the  sum  of  the  oj>timal  return  function  and  the  optimal 
accumulation  function  with  res]>ec.t  to  tlie  states  and  parameters  at  the  current 
time.  This  then  resulted  in  a  compensator  structure  which  is  both  robust  in 
that  it  chooses  a  control  based  on  the  worst  case  disturbances  and  parameter 
uncertainties  and  adai:)tive  in  that  the  uncertain  imrameters  are  estimated  using 
available  measurements. 

The  problem  of  flight  control  design  at  high  angles  of  attack  presents  a 
natural  application  of  such  robust  and  adaptive  compensators.  In  [4],  a  robust 
controller  was  designed  for  higli  angle  of  attack  flight  conditions  of  the  F-18 
HARV  (High  Angle-of- Attack  Research  Vehicle)  aircraft.  The  compensator  was 
designed  for  zero  steady  state  tracking  of  pilot  inputs  by  augmenting  the  state 
space  with  integral  error  states.  Additionally,  flue  to  the  physical  limitations 
of  the  thrust  vectoring  hardware  on  tlie  aircraft,  additional  “washout”  states 
were  added  so  that  tlirust  veettor  commands  were  faded  to  zero  in  steady  state. 
The  robust  compensator  design  was  tlien  used  to  effectively  expand  the  usable 
region  of  the  linear  controller  about  each  design  point. 

One  difficulty  which  arises  with  such  a  design  is  that  parameters  in  the 
linearized  system  may  change  rapidly  and  in  some  cases  switch  signs  over  dy¬ 
namically  varying  flight  conditions.  By  estimating  the  parameters  which  tend 
to  vary  the  most  and/or  have  the  greatest  effect  on  system  performance,  it  may 
be  possible  to  increase  the  overall  jxuTormance  of  the  compensator  as  parameter 
values  be(t)me  more  well  known.  The  unique  advantage  of  a  robust  adaptive 
compensator  such  as  that  which  is  ju'esented  in  this  paper  is  that  by  forming 
the  control  leased  on  the  worst  case  values  of  state  and  parameters,  the  com¬ 
pensator  can  effectively  use  the  controls  whose  coefficients  are  better  known 
until  enough  measurements  have  been  taken  to  reduce  the  uncertainty  in  the 
unknown  coefficients  to  the  point  where  the  associated  control  can  be  used  with 
confidence. 

In  the  exami^le  presented  in  this  paj^er,  the  longitudinal  dynamics  of  the  F- 
18  HARV  are  considered  at  a  given  flight  condition.  The  plant  states  are  angle 
of  attack,  a,  and  pitch  rate,  cy.  The  controls  to  be  used  are  elevator  deflection, 
6e,  and  thrust  vectoring,  The  i>roblem  is  to  follow  step  commands  in 

angle  of  attack  with  zero  steady  state  tracking  error  while  fading  the  thrust 
vectoring  commands  to  zero  in  steady  state  due  to  the  hardware  restrictions 
of  the  paddles  used  for  thrust  vectoring  on  the  F-18  HARV.  To  demonstrate 
the  behavior  of  the  robust  adaptive  compensator,  the  moment  coefficient  due  to 
elevator  deflection,  is  considered  to  be  unknown  and  is  to  be  estimated  on 
line.  Results  are  shown  which  illustrate  the  effec.t  of  the  disturbance  attenuation 
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bound. 


2  Disturbance  Attenuation  Problem 

The  problem  of  disturbance  attenuation  is  one  of  finding  a  control  which  limits 
the  effects  of  all  admissable  disturbances  and  uncertainties  on  the  compensated 
system.  A  disturbance  attenuation  function  is  formed  which  is  essentially  a  ratio 
of  the  norms  of  performance  outputs  over  disturbance  inputs.  The  problem, 
then,  is  to  find  a  positive  i^arameter  0  such  that  this  disturbance  attenuation 
function  is  bounded.  This  function  can  be  written  as: 


D  =  -g  < I  >0  (1) 

where  the  measures  of  i>erformance  outj^uts,  ||i7||'^  and  |hi)||^  are  defined  as 

llwll'  =  lN(V)llo,  +  /' +  (2) 

INI'  =  11^(0)11^'+/*' +  (3) 

where  x  reju'esents  the  states,  u  the  controls,  w  the  plant  input  disturbance,  v 
the  state  lne^Lsurenlent  noise,  and  ^  represents  the  augmented  state  defined  as 
^  =  [  ]^,  where  /?  reiM*esents  tlie  unknown  control  coefficient  matrix 

parameters.  The  dynamic  system  under  consideration  is  of  the  form: 


X  =  Ax  +  B  (/?)  w  -h  r?/;  (4) 

=  Hx  -\-v  (5) 

(0) 

where 

h 

B{/3)  =  +  (7) 

j=l 

P  =  0  (8) 

To  approach  this  problem,  as  in  [2,  3),  we  reformulate  the  disturbance  atten¬ 
uation  problem  as  a  differential  game  problem,  with  performance  index  given 
by; 


J  = 


(9) 
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For  a  given  value  of  0,  then,  the  problem  becomes  one  of  finding  the  control 
u  which  minimizes  this  cost  in  the  presence  of  the  worst  case  maximizing  distur¬ 
bance  inputs  provided  by  initial  conditions,  $(0),  and  state  and  measurement 
noise  (?/;,  . 

3  Dynamic  Programming  Solution 

It  is  shown  in  that  the  minimax  problem  associated  with  the  performance  index 
given  by  (9)  reduces  to  a  saddle  point  problem  so  that  the  operations  of  mini¬ 
mization  and  maximization  can  be  interchanged  [3,  5].  This  leads  to  a  dynamic 
programming  solution  with  an  accompanying  decomposition,  which  is  further 
developed  in  [3).  The  first  part  of  this  decomposition,  a  control  subproblem, 
defines  an  optimal  return  function,  which  represents  the  cost  to  go 

from  current  time,  t,  to  final  time,  t / .  The  second  part,  a  filtering  subproblem, 
defines  an  optimal  accumulation  function,  T(a;t,^),  which  represents  the  cost 
accumulated  from  initial  time,  0,  to  current  time,  t.  The  dynamic  programming 
problem,  then,  is  to  find  the  values  of  state  and  parameters,  x,  ^  such  that: 

T  (xt,  +  <1'  (.T,,  >  T  (xt,  /?)  +  (xt,  /?)  Vx,,  p  (10) 

3.1  The  Control  Subproblem 

The  control  subprf)l)lein  reineseiits  tlie  ta.sk  of  finding  the  control,  u,  which 
minimizes  the  cost  to  go  from  the  current  time,  t,  to  the  final  time,  tj.  To 
accomplish  this,  we  first  write  this  cast  as: 

Jc  =  I  |||•'c(t/)||Q^  +  I^HxIIq  +  ||w1|r  -  i  (||w||^-i  +  lkllv->)j  ‘^'^1  (11) 

Appending  the  dynamics  (4)  to  the  cost  and  solving  as  in  [3]  yields  an 
expression  for  tlie  minimizing  control,  w,  and  maximizing  disturbance,  w,  as  a 
function  of  the  states  and  parameters.  The  parameter  dependence  arises  through 
the  associated  R.icc:ati  etiuation,  which  is  calculated  using  the  control  coefficient 
matrix,  B  (/3)  evaluated  for  the  given  values  of  the  parameters,  p.  Thus,  we 
have: 


w(r)  =  -R-^B(pfYl{P,r)x{r)  (12) 

w(t)  =  Wr^n(/?,T)x(T)  (13) 

where  the  matrix  11  (/?,  r)  is  defined  as  the  solution  of  the  Riccati  equation 

-li  =  A’^Y\-\-UA-\-Q-n(B{P)R-^B{pf' -eTWV'^')n  (14) 
n(t/)  =  Qj 
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Substituting  the  expressifnis  obtained  for  tlie  control  and  disturbance  back  into 
the  performance  index  (11)  gives  the  desired  expression  for  the  optimal  return 
function: 


^{xt,l3)  =  ^xjTl{P)xt  (15) 

The  control  which  arises  from  this  portion  of  the  solution,  then,  is  based 
upon  full  information  of  the  state  and  parameter  values.  This  solution  has  been 
further  extended  to  allow  infinite  final  time  in  [5]. 

3.2  The  Filtering  Subproblem 

The  second  part  of  the  problem  considered  is  that  of  solving*  the  optimization 
problem  in  reverse  time  from  time  0  to  the  j^resent  time,  t.  To  do  this,  the  cost 
is  first  written  in  terms  of  the  augmented  state, 


=  I  {-1 11^  +  Jo  ~  I 


where 


0  = 


Q 


and  the  dynamics  of  the  system  (4)  are  rewritten  in  terms  of  the  augmented 
state,  ^  as: 


^  =  A^  +  Bu-\-rti)  (17) 

z  =  (18) 

with  i4,  By  f ,  and  H  defined  as 


A  Biu  •  •  •  Bfcu 
0  0 


B 


H^[H  0  ] 


Adjoining  the  augmented  dynamics,  (17)  to  the  performance  index  (IG)  and 
solving  the  o])tiinization  problem  results  in  an  equation  for  an  estimator  which 
gives  estimates  of  the  state  and  i>arameter  values  at  time  t  based  on  the  in¬ 
formation  contained  in  the  measurements,  z,  up  to  time  t.  This  estimator  is 
represented  by: 


=  {A  +  0PQ)^  +  Bu  +  PH'^V-^{z-H^)  (19) 

^(0)  =  $0 
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where  the  matrix  P  is  the  solution  of  a  Rictcati  equation  given  by: 


P  =  AP  +  PA^ +  fwf'^ -P{H'^V-^H -eQ)P  (20) 

P(n)  =  Po 

Combining  these  relations  witli  the  j>eiformance  index  as  in  [3]  gives  the 
desired  relationship  for  the  optimal  accumulation  function: 

T{xt,l3)  =  -^e^5e  (21) 

where 

y  =  \  {'<ARn  +  -  ^  (-^  -  Hxf  V-i  {z  -  Hx) 

e  =  it -it 

s  =  pr' 

3.3  The  Connection  Condition 

The  final  piece  in  forming  the  adaptive  compensator  is  to  perform  the  maxi¬ 
mization  of  the  sum  of  the  optimal  return  function  (15)  and  the  optimal  accu¬ 
mulation  function  (21)  at  time  t.  That  is,  we  wish  to  find  the  worst  case  state 
and  i^arameter  values,  :v*  and  wliicdi  are  found  from  the  maximization 


max[^/(x,,/?)  +  T(xt,/?)]  (22) 

This  maximization  can  be  simi^Iified  somewhat  by  first  solving  the  maxi¬ 
mization  with  resi)ect  to  the  state  xt  which  gives  an  algebraic  relation  as  a 
function  of  the  parameter  (3.  Partitioning  the  matrix  S  as 


5  = 


Sxx  Sx.p 


the  expression  for  tlie  worst  cjuse  state  a;*  as  a  function  of  the  parameter  j3  is 
then  given  by 


xl  =  (OT  (/?)  -  (p-p)-  S^^xt 

with  the  requirement  that 


(23) 


6)n(/?)-5*x<0  V/? 


(24) 
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The  maximization  problem  can  then  be  solved  as  a  function  of  the  parameter 
only.  It  should  be  noted,  however,  that  since  the  Riccati  solution  Il(/3)  is 
dependent  on  /?,  the  resulting  function  to  be  maximized  becomes  quite  nonlinear 
and  may  have  more  than  one  local  maximum.  It  is  possible  that  at  some  time  t 
there  could  be  two  i>eaks  of  ecjual  magnitude.  Should  such  a  case  arise,  it  should 
be  noted  that  although  the  worst  case  state  and  parameter  values  at  these  peaks 
will  be  different,  the  resulting  control  will  be  the  same  [3]. 


4  Application  to  Longitudinal  Flight  Control 

The  basic  dynamic  system  to  be  considered  in  this  example  is  a  system  consisting 
of  two  states  (angle  of  attack,  a,  and  pitch  rate,  q)  and  two  controls  (elevator 
deflection,  6c,  and  thrust  vector  command,  6tvc)‘  The  basic  dynamics  for 
this  system  are  obtained  by  linearizing  the  dynamics  of  the  airplane  about  a 
particular  trim  condition.  For  this  j)articullar  example,  a  flight  condition  at  an 
altitude  of  25,000  feet  and  angle  of  attack  of  10  degrees  was  selected.  The  basic 
dynamic  system,  then,  can  be  written  in  the  form: 


d 


1  0 
0  1 


a 

<1 

H- 


-f 


^^TVC 


Se 

6tvc 


(26) 


In  formulating  the  dynami(^  system  to  be  controlled,  we  first  consider  the 
objec'.tives  of  the  cunitrol  design.  Primarily,  we  would  like  to  be  able  to  track  step 
commands  in  angle  of  attack,  a,  with  zero  steady  state  error.  As  a  secondary 
objective,  we  would  like  to  fade  the  thrust  vector  control  command,  Stvci  to 
zero  in  steady  state  to  avoid  damaging  the  jjafldles  which  are  used  as  actuators 
for  thrust  vec^toring  on  the  F-18  HAKV.  Also,  since  the  control  effectiveness  of 
the  elevator  can  cliange  at  varying  flight  conditions,  we  consider  Ms^  as  the 
uncertain  ]:)arameter  to  be  estimated  on  line. 

To  ac-complish  our  first  objective,  we  formulate  a  change  of  variables  and 
define  the  error  coordinate  as  the  error  between  actual  angle  of  attack,  a, 
and  commanded  angle  of  attack, 


Ca  ”  o  “  Olc  (27) 

To  track  step  commands  in  a  with  zero  steady  state  error,  a  constant  value 
of  the  control  deflection  6c  will  be  required  in  steady  state.  To  assure  that  the 
problem  remains  well  posed  we  must  form  a  new  state  space  by  differentiating 
the  error  so  that  the  control  in  the  dynamic  system  used  in  the  design  synthesis 
is  actually  the  derivative  of  the  actual  physical  control.  This  assures  that  we 
have  a  state  spacte  for  which  the  performance  index  will  remain  finite  as  final 
time  becomes  infinite  [6]. 
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Next,  we  need  to  incorporate  a  means  of  fading  the  thrust  vector  command, 
Stvc^  to  zero  in  steady  state.  To  do  this,  we  include  Stvc  in  the  state  space 
which  we  can  then  weight  in  our  performance  index  so  that  it  is  driven  to  zero 
in  steady  state.  However,  we  must  also  note  that  the  system  must  remain 
controllable  for  all  values  of  the  parameter  (3,  To  assure  that  this  condition  is 
true,  we  redefine  controls, 


known  "i” 

where  {^e)known  niultiplied  by  a  fixed  value  of  the  control  coefficient,  {Ms^)q 
which  allows  the  system  to  remain  controllable  for  all  values  of  the  unknown 
value  of  which  multiplies  Since,  in  reality,  the  value  of  Ms^  is 

uncertain,  we  choose  tlie  ciontrol  weiglitings  in  the  performance  index  so  that  the 
cost  associated  with  using  is  much  greater  than  the  cost  associated 

with  using 

The  dynamic  system  used  in  the  solution  of  the  control  subproblem,  then, 
is  defined  as  in  (4)  with 


■  (4) 
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For  the  filtering  subi>rf)blem,  tlie  augmented  state  is  simply 


As  an  additional  iK)te,  since  for  the  objectives  we  have  specified,  the  pitch 
rate  q  does  not  need  to  be  controlled,  we  effectively  ignore  that  state  when 
formulating  the  controller.  To  be  able  to  control  q  we  would  either  need  to 
include  another  aerodynamic  control  such  as  flaj:)  deflection,  or  allow  the  thrust 
vectoring  command  to  attain  a  nonzero  steady  state  value. 

In  solving  the  maximization  jnol^lem  to  o!>tain  the  connection  condition,  we 
note  that  the  state  Stvc  is  acttually  something  which  we  calculate  directly  and 
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its  dynamics  are  decf)vi])led  from  the  rest  of  the  states,  so  that  we  may  simply 
adjoin  the  ctonstraint  brvc  =  to  (22).  Partitioning  the  Riccati  matrix  11 
as 

rr  _  Hgia;  llx-it 

n  n 

the  connection  condition  for  the  worst  case  state  becomes 

X*  =  [OTa-x  (/5)  “  ~  ”  OUxu^TVC^  (28) 

5  Simulation  Results 

To  demonstrate  the  behavior  of  tlie  adai^tive  controller  at  varying  values  of  the 
parameter  0,  step  resixnisas  in  angle  of  attack  were  simulated.  To  emphasize 
the  effect  of  0  on  the  l)ehavi()r  of  the  compensated  system,  the  initial  parameter 
estimate  was  taken  to  have  the  o])j)osite  sign  of  the  true  value  of  Ms^ .  A  step 
input  of  10  degress  from  tlie  initial  trim  condition  at  10  degrees  angle  of  attack 
and  25,000  feet  altitude  was  c*ommanded.  The  linearized  system  coefficients  at 
this  flight  condition  are  given  in  Table  1.  To  best  demonstrate  the  effect  of  the 
adaptive  controller,  the  initial  estimate  of  wtis  taken  to  be  2.0. 


Coefficient 

Value 

2a 

d).33G7 

2,; 

().907G 

M,. 

-().2()()5 

M., 

-0.1229 

Zk 

-0.0093 

^^TVC 

-0.0278 

Mk 

-2.7320 

hd^TVC 

-1.4747 

Table  1:  Linearized  System  Coefficients,  a  =  10  deg,  h  =  25,000  ft 

In  Figure  1,  10  degree  step  responses  in  angle  of  attack  are  presented.  In  the 
case  of  6?  =  0,  the  response  initially  moves  in  the  opi^osite  direction  before  rising 
to  the  commanded  angle  of  attack  and  eventually  settling  to  a  zero  steady  state 
error.  As  6  is  increased,  heiK:e  the  disturbance  attenuation  bound  decreased, 
the  step  resjionses  become  fluster  and  no  longer  exhibit  the  characteristic  of  an 
initial  response  in  the  negative  direction.  This  is  also  reflected  in  the  pitch  rate 
responses  shown  in  Figure  2. 

To  understand  why  this  increruse  in  0  improves  the  step  response,  we  can 
examine  the  control  cuunmands.  First,  examining  the  elevator  deflection  rate, 
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shown  in  Figure  .3  and  elevator  deflection,  Sc^  shown  in  Figure  4  we  see 
that  for  ^  =  0  a  great  deal  of  effort  is  applied  using  this  control,  although  it 
is  actually  moving  in  the  wrong  direction  due  to  the  erroneous  initial  estimate 
of  Ms^ .  As  ^  is  increased,  though,  we  see  that  the  controller  initially  uses  very 
little  elevator.  Conversely,  in  Figures  5  and  G,  we  see  that  at  0  =  0  that  a 
relatively  small  amount  of  thrust  vectoring  is  used,  but  as  ^  is  increased,  thrust 
vectoring  is  used  more  heavily  initially  before  fading  to  zero  in  steady  state.  In 
essence,  the  controller  seems  to  hedge  against  using  the  elevator  initially,  due 
to  its  uncertain  effects,  while  relying  more  heavily  upon  the  thrust  vectoring, 
whose  effect  is  known  with  much  greater  f«rtainty. 

The  estimated  valuers  of  shown  in  Figure  7  reflect  the  heavier  utilization 
of  elevator  for  0  =  0.  As  0  is  increased  from  0,  the  estimator  response  is  initially 
slower,  but  becomes  faster  as  0  increases.  The  key  factor  in  the  behavior  of 
the  controller,  however,  lies  in  the  worst  case  parameter  value,  plotted  in 
Figure  8.  Initially,  the  true  value  of  the  parameter  is  highly  uncertain,  which 
is  reflected  in  the  “variance”  of  tlie  j^arameter  shown  in  Figure  9  and  the 
“cross  covarianc’.e”  between  the  ]^arameter  and  state,  as  shown  in  Figure  10.  At 
some  point,  as  the  ]^arameter  value  becomes  more  well  known,  this  worst  case 
value  begins  to  follow  the  estimate,  and  the  controller  then  begins  to  use 
the  elevator  deflection  more  confidently. 

The  ])oint  in  time  at  which  the  controller  begins  to  follow  the  estimated 
value  of  is  a  function  of  the  maximization  (22).  At  a  particular  point  in 
time,  it  is  po.ssible  for  this  function  to  have  more  than  one  local  maximum. 
For  0  =  5,  Figure  11  shows  that  this  sum  around  the  time  that  the  worst 
case  paramter  begins  to  follow  the  estimated  i)arameter  has  two  peaks.  At 
a  certain  point  in  time,  the  peak  whicii  more  closely  follows  the  parameter 
estimate  begins  to  dominate  as  the  deca-ejusing  variance  causes  perturbations 
from  the  estimated  value  to  be  weighted  more  heavily  At  this  point  in  time, 
the  worst  case  j^arameter  jumi)s  to  a  value  corresponding  to  this  peak.  Then, 
as  this  i)eak  begins  dominating  the  other  peak,  the  controller  begins  to  use  the 
elevator  deflection  nK)re  lieavily. 


6  Conclusions 

The  results  j^resented  in  this  pai>er  demonstrate  the  performance  of  a  robust 
adaptive  C’ompensator  based  on  disturbance  attenuation.  These  results  show 
that  as  the  disturbance  attenuation  bound  is  lowered,  the  compensator  tends 
to  rely  more  uj^on  controls  whose  effects  are  know  with  more  certainty  until 
estimates  of  uncertain  coefficients  are  known  with  enough  confidence  to  be  used 
effectively.  The  most  dramatic  improvements  in  performance  occur  when  the 
initial  estimates  of  the  unknown  ]>arameters  are  farthest  from  their  true  values. 

For  systems  which  c.ontain  a  great  deal  of  parameter  uncertainty  in  some  or 
all  of  the  coefficients,  such  jls  flight  control  systems,  this  type  of  compensator 
can  prove  to  be  very  useful.  Without  any  a  priori  restrictions  on  the  structure 
of  the  comi^ensator,  the  distuii)anc.e  attenuation  approach  results  in  a  design 
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which  is  both  robust  in  that  it  tends  to  herlge  against  uncertain  states  and 
parameters,  ami  adaptive  in  that  it  uses  the  measurement  history  to  update 
its  knowledge  of  the  unknown  parameters.  This  type  of  design  is  particularly 
useful  for  systems,  such  as  the  flight  control  system  examined  in  this  paper,  in 
which  parameters  may  vary  in  magnitude  and/or  sign  over  varying  conditions. 

By  using  this  robust  adaptive  compensator,  the  controller  not  only  has  the 
ability  to  update  its  information  of  the  system  model,  but  is  designed  in  such 
a  way  that  it  chooses  its  control  based  on  how  well  this  model,  or  parameters 
within  the  model,  is  known.  As  more  information  becomes  available  to  the 
controller,  the  jjarameters  within  the  model  become  more  well  known,  and  the 
controller  is  able  to  u.se  this  increased  certainty  in  the  system  model  to  utilize 
controls  which  are  mo.st  affected  liy  the  uncertain  coefficients  with  more  confi¬ 
dence.  By  using  the  controls  which  are  known  with  the  greatest  certainty,  the 
overall  performance  of  the  system  can  then  be  improved. 
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Figure  3:  Elevator  CoinniaiKl  Rate, 


Figure  4:  Elevator  Comniand,  6^ 
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TVC  Deflection  (deg)  TVC  Rate  (deg/sec) 


Figure  5:  Thrust  Vector  Coinniand  R^te,  Stvc 


Figure  G:  Thrust  Vector  Command,  Stvc 
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Worst  Case  Coefficient  Coefficient  Estimate 


Figure  7:  Estimated  Parameter  Value, 


Figure  8:  Worst  Case  Parameter, 
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Figure  10:  Paranieter/Pitch  Acceleration  Cross  Covariance, 


l(i 
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Figure  11:  Cost,  [^x'^Ux  -  ^e^5e]  As  a  Function  of  Ms^ 
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Approximate  Nonlinear  Game-Theoretic  Estimation  and 

Control 

Jinsheng  Jang  and  Jason  L.  Speyer 
University  of  California,  Los  Angeles 

Abstract 

For  a  class  of  linear  dynamical  systems  with  nonlinear  perturbation,  an  approximate 
optimal  estimation  scheme  is  derived  based  on  a  deterministic  game-theoretic  crite¬ 
rion.  Using  the  calculus  of  variation  approach,  the  process  disturbance  and  initial 
state  vector  are  first  maximized.  The  resulting  optimality  condition  is  then  expanded 
with  respect  to  a  small  parameter  to  solve  for  the  worst  case  state  and  the  Lagrange 
multiplier  term  by  term.  Subsequently,  the  approximate  optimal  estimator  is  de¬ 
rived  by  minimizing  a  series  of  cost  criterions  over  its  corresponding  state  estimate 
vectors.  The  estimator  Riccati  differential  equations  (RDE)  necessary  for  the  first 
and  higher  order  correction  terms  are  shown  to  be  the  same  as  in  the  zeroth-order 
case.  Then,  the  infinite  order  approximate  minimax  estimator  is  shown  to  be  dis¬ 
turbance  attenuating.  In  addition,  the  N-th  order  approximate  minimax  estimator 
is  proved  to  achieve  disturbance  attenuation  with  a  higher  threshold  proportional  to 
the  -|- 1  power  of  the  expansion  parameter.  Therefore,  as  more  and  more  terms 
are  added,  the  N-th  order  threshold  moves  closer  and  closer  to  the  infinite  case.  For 
a  similar  class  of  linear  dynamical  system  with  similar  perturbation,  an  approximate 
game-theoretic  measurement  feedback  controller  is  obtained.  As  in  the  estimation 
problem,  first  a  maximization  problem  with  respect  to  the  disturbance  and  initial 
state  vector  is  performed.  Then,  a  minimax  deterministic  game  is  considered  where 
the  measurement  noise  is  acting  as  a  maximizing  player  and  the  control  is  treated  as 
a  minimizing  player.  The  analytical  form  of  the  zeroth-order  term  for  the  approx¬ 
imate  controller  resembles  the  linear  solution.  However,  the  first  and  higher  order 
correction  terms  of  the  controller  require  two  estimator  where  one  of  the  estimator 
is  almost  the  same  as  in  the  pure  estimation  case.  The  other  estimator  is  derived 
from  a  two-point  boundary  value  problem  associated  with  the  control  minimization 
sub-problem.  For  the  zeroth-order  case,  the  two  RDEs  for  the  control  problem  are  the 
same  as  in  the  linear  case.  Although  the  estimator  RDE  remains  the  same  as  the  order 
of  the  problem  progresses,  the  controller  RDE  for  the  first  and  higher  order  problem 
is  different  from  the  zeroth-order  case.  Finally,  the  approximate  game-theoretic  con¬ 
troller  is  proved  to  have  disturbance  attenuation  property  when  infinite  order  terms 
are  used.  Furthermore,  this  property  is  kept  when  only  finite  correction  terms  are 
used  for  the  approximate  controller  using  an  increased  threshold  value  proportional 
to  the  ^  -t-  1  power  of  the  expansion  parameter.  As  expected,  this  threshold  moves 
closer  and  closer  to  the  infinite  order  threshold  as  more  and  more  correction  terms 
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are  utilized  in  the  approximate  controller  and  recovers  the  original  threshold  when 
infinite  correction  terms  are  incorporated. 

I.  Introduction 

In  the  literature,  the  research  in  the  area  of  the  deterministic  approximate  op¬ 
timal  guidance  has  been  well  documented  [1,  2,  3].  Two  approaches,  namely  the 
Hamilton-Jacobi-Bellman  equation  expansion  technique  [1]  and  the  calculus  of  varia¬ 
tion  approach  [2,  3],  are  related  to  our  works  here.  Surprisingly,  very  few  reports  on 
the  equally  important  approximate  optimal  estimation  problem  using  the  above  two 
deterministic  approaches  have  been  investigated.  In  addition,  the  disturbance  atten¬ 
uation  property  of  the  approximate  guidance  scheme  as  derived  in  [1,  2,  3]  has  not 
been  studied.  To  the  authors’  knowledge,  only  the  stochastic  approximate  optimal 
estimation  using  also  a  power  series  expansion  technique  is  found  in  [4].  In  [4],  for 
a  specific  polynomial  nonlinearity  of  order  3,  an  approximate  estimation  scheme  is 
derived  by  indirectly  calculating  the  approximate  conditional  density  function  using 
perturbation  method.  Their  results  are  only  valid  for  a  scalar  system.  Furthermore, 
in  [4],  explicit  formula  to  determine  the  expansion  terms  of  the  conditional  density 
is  obtained  only  for  a  very  simple  example.  Recently,  in  [5],  a  deterministic  game 
problem  as  here  is  formulated  for  a  singularly  perturbed  dynamical  system.  Only  the 
linear  dynamical  system  is  treated  in  [5]. 

In  this  work,  using  a  regular  perturbation  approach,  for  a  class  of  linear  dynamical 
system  with  nonlinear  perturbations  in  both  system  dynamics  and  measurement  equa¬ 
tions,  an  approximate  estimation  scheme  is  derived.  A  calculus  of  variation  approach 
[6]  is  used  to  derive  the  estimator  based  on  a  deterministic  game-theoretic  perfor¬ 
mance  index.  First,  by  expanding  a  nonlinear,  two-point  boundary  value  problem 
with  respect  to  the  small  parameter,  a  series  of  linear,  solvable  two-point  boundary 
value  problems  are  obtained  and  solved  for  the  worst  case  state  and  Lagrange  multi¬ 
plier  vectors  term  by  term.  Next,  setting  the  small  parameter  to  zero,  the  zeroth-order 
solution  for  the  approximate  filter  is  shown  to  be  the  linear  Hoc  estimator  as  derived 
in  [7].  Furthermore,  first  and  the  higher  order  correction  terms  for  the  approximate 
game-theoretic  estimator  are  derived  via  the  minimization  of  a  series  of  performance 
indexes  with  respect  to  its  corresponding  state  estimate  correction  terms.  The  per¬ 
formance  indexes  are  results  of  the  power  series  expansion  of  the  original  performance 
index  with  respect  to  the  same  small  parameter.  Since  only  the  filtering  problem  is 
of  interest  here,  the  resulting  estimation  equation  is  simplified  by  substituting  the 
boundary  condition  of  the  Lagrange  Multiplier  into  its  associated  optimality  condi¬ 
tion.  The  RDEs  for  the  higher  order  correction  terms  are  derived  to  be  the  same  as 
the  RDE  in  the  linear  Hoo  problem. 

Subsequently,  the  disturbance  attenuation  problem  as  defined  in  section  II  is 
shown  to  be  solved  by  this  infinite-order  approximate  game-theoretic  filter.  Fur¬ 
thermore,  the  N-th  order  disturbance  attenuation  problem  is  proved  to  be  solved  by 
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the  N-th  order  approximate  game-theoretic  filter  which  is  truncated  from  the  infinite 
order  filter.  Particularly,  the  disturbance  attenuation  threshold  for  the  approximate 
game-theoretic  filter  of  arbitrary  order  is  shown  to  be  upper  bounded  by  twice  the 
original  threshold  and  the  increase  in  the  threshold  is  proportional  to  the  N  +  1 
power  of  the  expansion  parameter.  Therefore,  as  the  order  of  the  estimator  increases, 
the  threshold  decreases  and  reduces  to  the  original  threshold  as  the  order  approches 
infinity. 

Next,  the  approximate  game-theoretic  control  is  formulated  and  derived  by  solving 
a  disturbance  attenuation  problem  with  a  regular  perturbation  approach  bu  using 
the  calculus  of  variations.  The  derivations  are  separated  into  two  parts.  First,  a 
maximization  problem  is  solved  with  respect  to  the  disturbance  and  initial  state 
vectors.  Following  that,  the  first-order  necessary  conditions  are  derived.  Similarly  as 
in  the  estimation  case,  a  nonlinear  two-point  boundary  value  problem  is  formed  based 
on  the  original  system  dynamics  and  the  first-order  optimality  conditions.  Naturally, 
both  this  nonlinear  two-point  boundary  value  problem  and  the  cost  function  are 
expanded  using  a  combination  of  power  series  expansion  and  Taylor’s  series  expansion. 
Secondly,  a  series  of  linear  minimax  deterministic  game  problems  with  respect  to  the 
expansion  terms  of  the  cost  function  are  solved  sequentially  again  using  a  calculus  of 
variation  approach.  In  each  problem,  the  first  order  necessary  condition  is  derived, 
then  another  two-point  boundary  value  problem  is  formed  based  on  the  necessary 
condition. 

The  zeroth-order  solution  of  the  approximate  controller  resembles  the  linear  case 
as  obtained  in  [9].  The  first  order  and  higher  order  correction  terms  of  the  controller 
require  two  estimator.  The  first  estimator  is  almost  the  same  in  the  pure  estimation 
case.  The  second  estimator  is  derived  from  the-  two-point  boundary  value  problem 
from  the  control  minimization  part  of  the  problem  solution.  For  each  expansion  term, 
the  controller  requires  the  solutions  of  two  RDEs,  namely  the  estimator  RDE  and  the 
controller  RDE.  The  estimator  RDe  remains  intact  as  the  order  of  the  correction 
terms  progresses.  However  the  controller  RDE  for  the  first  and  higher  order  case  is 
different  from  the  zeroth-order  solution.  This  is  due  to  the  fact  that  the  measurement 
vector  is  a  given  sequence  and  thus  not  expanded.  Therefore,  the  measurement  vector 
is  treated  as  a  zeroth-order  term  only.  As  expected,  similar  disturbance  attenuation 
results  are  proved. 

The  present  paper  is  organized  as  follows.  In  section  II,  the  disturbance  atten¬ 
uation  problem  for  the  approximate  game-theoretic  estimation  is  folumated.  The 
problem  is  then  solved  using  a  game-theoretic  approach.  Using  a  caculus  of  varia¬ 
tion  technique,  the  first  order  necessary  condition  is  derived  for  the  maximization 
problem.  In  section  III,  the  zeroth-order  solution  is  obtained  solving  a  minimization 
problem  with  respect  to  the  zeroth-order  term  of  the  cost  function.  The  first  order 
and  higher  order  correction  terms  for  the  approximate  estimator  are  derived  in  sec¬ 
tion  IV.  Section  V  presents  the  results  of  the  disturbance  attenuation  property  of  the 
estimator.  Finally,  the  approximate  game-theoretic  controller  for  a  output  feedback 
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control  problem  is  derived  in  section  VI. 

11.  Problem  Formulation 

Consider  a  linear  dynamical  system  with  nonlinear  perturbation 

X  =  Ax  +  tg{x)  +  Tiw  (1) 

and  the  measurement  equation  with  nonlinear  perturbation  as 

z  =  Hx  +  eh{x)  +  V  (2) 

where  xtR^^  and  veR^  are  the  state,  process  disturbance,  and  the  measurement 

noise  vectors,  respectively.  In  this  paper,  g{x)  and  h{x)  are  assumed  to  be  continuous 
and  infinitely  differentiable.  To  start,  denote  a  disturbance  attenuation  function  as 


/o  11^  -  i|l«* 


i|x(0)  -  i(0)E.,  +  f‘  IhllJ,..,  +  11^  -Hx-  (h{x) 


2  dr 

y-lO,T 


(3) 


where  x  is  the  state  estimate  vector.  The  numerator  and  the  denominator  of  the 
disturbance  attenuation  function  are  measures  of  the  output  and  input  energy  using 
quadratic  norm.  Based  on  (3),  the  disturbance  attenuation  problem  is  defined  as  to 
find  an  optimal  estimator  i*(-)  such  that 


Daf{x*^w,x{0))  <  0,  0>0  (4) 

for  all  weL2[0,t]  and  a;(0)€i?"  such  that  (i(;(r),u(r))  ^  0  for  all  Tc[0,t]  and  a;(0)  ^  ®(0). 
In  (4),  6  is  called  the  disturbance  attenuation  threshold,  ®*(*)  is  defined  as  a  causal 
mapping  as 


x-(r)  =  ®[Z(t)1  (5) 

where  Z(t)  =  {z(s)|0  <  s  <  r}  is  the  measurement  history.  Intuitively,  0  can  be 
interpreted  as  a  upper  bound  of  a  transfer  function  measuring  the  effect  of  the  input 
norm  upon  the  output  error  norm. 

To  solve  the  disturbance  attenuation  problem  as  defined  in  (4),  we  adopt  a  mini¬ 
max  approach.  A  seperable  performance  index  is  obtained  from  (4)  as 

J(i;ti),x(0))  =  -^||a;(0)  -  x(0)||5,-,  +  ^  i||a:  -  -  ^[||u;||^_, 

-t-ll^r  -  Hx  —  th\\y-i]dT  (6) 
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The  problem  is  to  find  an  estimation  scheme  based  on  the  optimization  of  the  per¬ 
formance  index  in  (6)  for  the  system  described  as  in  (1)  and  (2).  Thus,  consider  the 
following  deterministic  game 


minimax J,  (7) 

where  x{-)  is  trying  to  minimize  J  while  the  adversaries,  a;(0)  and  w{-),  are  intending 
to  maximize  J.  J  is  as  chosen  in  (6)  subject  to  constraints  (1)  and  (2)  which  is 
explicitly  introduced  into  the  cost  J  using  a  Lagrange  Multiplier  A.  Note  that  the 
maximization  of  the  measurement  noise  is  implicit  through  the  constraint  (2)  since 
the  measurement  vector  z  is  given  and  the  maximization  of  the  initial  state  vector 
x(0)  and  the  process  disturbance  w  leaves  no  more  freedom  to  the  measurement  noise. 

The  maximization  procedure  is  tackled  first.  For  our  conviences,  denote 

J  (^7  ^  ^  (^) 

From  the  calculus  of  variation  approach  [5],  the  first  order  necessary  conditions  of 
optimality  are  easily  obtained  as 


A^(0)  =  [x(0)  -  x(0)^Po■^  A^(t)  =  0, 

A^  +  X^cg:,  +  X'^A  +  {z-Hx-  ehfV-\H  +  eK)  +  O'^x  -  xfQ  =  0 

-H  A^r  =  0  (9) 

where  the  partial  derivativve  matrices  are  defined  as  and  gx  =  From 

(9)  and  (1),  a  nonlinear  two  point  boundary  value  problem  is  formed  as 


X 

A  rVFF^ 

X 

X 

{H  +  ehxfV-^H  -  -A^  -  e{gxf  . 

X 

+ 


iH  +  ehx)^V-\z-eh)  +  0-^Qx 


,  x(0)  =  f  (0) -f  PoA(O),  A(t)=:0  (10) 


II.  1  An  Ordinary  Perturbation  Approach  to  the  Solution  of  the 
Disturbation  Problem 


In  general,  it  is  impossible  to  solve  for  the  worst  case  x  and  A  analytically  from 
(10).  Therefore,  the  worst  case  x  and  A  are  determined  approximately  by  expanding 
(10)  in  a  power  series  [2].  Note  that  2:  is  a  given  data  sequence  and  is  not  expanded 
below.  To  start  the  expansion,  let 

00  00  00 

X  =  A  =  ^v,  i  =  (11) 

j=o  i=o  j=o 
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Subsequently,  g,  gx,  h,  and  hx  are  expanded  using  Taylor’s  series  expansion  with 
respect  to  iq  as 


where 


g{x)  =  g  +  gx{x  -  xq)  +  -{x  -  xofgxx{x  -  xo)  +  •  •  • 

=  go  +  ^9i  "t"  ^^92  4-  •  •  • 

9xix)  =  9x0  +  eflfri  +  ^^9x2  +  •  •  • 


90 

91 

92 


=  9  =  9{x)\x=xo 
—  9x^1  —  |x=zo^l 

_  -  It-  _  ^5(^)1  1  t^^9{^)\ 

=  9x^2  "1“  2^1  Sxx^l  =  |x=xo^2  “h  2^1  ^^2 


(12) 

(13) 


(14) 


d9 

1 

9x0 

III 

II 

§^1 

|x=Xo 

d^9. 

9xi 

—  9xx^l  — 

9X2 

=  9xx^2  “I" 

1  T- 

2^1 9xxx^i 

Note  that  9xxi9xxxi  etc.,  are  tensors.  For  example,  for  our  conveniences,  denote 


(15) 


{x  To)  9xx(^X  To)  — 


(t  -  To)^^l, 


(t  -  To) 


(16) 


(t  -  To)^^n,®® 


as  a  column  vector  instead  of  using  tensor  notation.  Similarly,  h{x)  and  hx{x)  can  be 
expanded.  The  related  notation  is  self-explanatory.  Equation  (10)  is  then  expanded 
using  (11),  (12),  and  (13).  After  equating  the  coefficients  of  like  powers  of  e,  a  series 
of  linear,  two-point  boundary  value  problems  are  obtained  as 


e 


1  . 


Xq 

A  riFr’’  ■ 

To 

.  ^0. 

-  e-^Q)  -A^ 

.^0. 

^o(O)  ~  ^(0)  ^o^o(0)j  0 

■  Ti 

A  VWT^  ' 

Ti  ■ 

.  . 

-  e-^Q)  -A^ 

.^1 . 

0 

-H^V-'^z  +  O-'^Qxo 

(17) 
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xi(0)  = 


2  ^2 
'  '  A2 


[  -hl,V-\z  -  Hxo)  +  H^V-^ho  -  glXo  +  O'^Qx,  J  ’ 

■fb-^i(O);  Xi{t)  =  0 

A  Twr^  U  ■ 

-  e-'^Q  -A^  X2 


2^2(0)  =  PoA2(0);  A2(0  =  0 


where 


g,  =  -klV-'(,z-Hx„)  +  hlV-'(h„  +  Hx,)-gl\o-glX, 

+H'^V-^hi  +  e-^Qx2.  (20) 

Let  Xj  and  Ay  be  the  solutions  to  the  corresponding  problems  in  (17)-(19).  Denote 


x-  =  ^x,V.  V  =  ^A,V. 


Similarly, 


=  «*-(^')  =  E4'''  K*')  =  E '■i''’  *.-(*')=E*^/'  (22) 


Furthermore,  J*  is  expanded  as 


r  = 


a  °°  /"M  °°  °° 

-5iiE^<'('>)«'iift+  /  2ii(E^i'^)-(E^i^’)ii« 

j=o  j=o  i=o 

„  00  „  00  00 

-5II E  -  5lk  -  ff(E  -  <12 


Denote  J*  =  Thus 


Jo  =  -X 


Ji  = 


2ll'^o(0)llpo  2^1®°  ~  ~  ~  2^^^  ~  (24) 

^[AS(0)f  PoAt(O)  +  /Vs  -  (^o)^g(a)I  -  x^)  -  ^[ASl^FlTr^AJ 
Jo 
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(25) 


-e{z-HxlfV-\-Hxl-hl)dT 

J2  =  —  2  j  (®0  ~  ^0)  Q{X2~X2) 

+i|k:  -  iill«  -  <ilA;Frirr’'A;  -  j||A;i|f„,r, 

-e(z  -  Hx'ofv-'i-Hx;  -  K)  -  jiiifi;  +  h’„\\\,-,dT  (26) 

Note  that  the  e°  terms  in  (17)  and  (24)  are  exactly  the  same  as  in  the  linear  case  [6]. 
Thus,  zeroth-order  solution  is  indeed  the  linear  H^o  estimator  which  will  be  derived  in 
section  III  by  minimizing  Jq  with  respect  to  xq  subject  to  its  associated  constraints. 
As  shown  in  section  IV,  the  first  order  correction  term  for  the  estimator  will  be 
obtained  by  minimizing  J2  with  repect  to  the  first  order  correction  term  of  the  state 
estimate,  namely,  xi  subject  to  its  associated  constraints.  Based  on  this  philosophy 
of  the  perturbation  method,  the  higher  order  correction  terms  of  the  estimator  are  to 
be  determined  in  section  IV. 

III.  Zeroth- Order  Solution  -  the  Linear  H^o  Estimator 

To  derive  the  zeroth-order  solution,  consider  the  linear,  two-point  boundary  value 
problem  as  in  (17).  Assume 


x-^x^  +  PX;  (27) 

where  Xc^  is  an  intermediate  variable  and  P  is  the  Riccati  variable,  both  of  which  are 
to  be  evaluated  later.  Differentiation  of  the  above  equation  and  substitution  of  both 
sides  of  (17)  yields 


[AP  +  TWT^  -P-  PiH^V-^H  -  e-'^Q)P  +  PA^]X; 

=  -Axc,  ico  +  P{H^V-'^H  -  0-^Q)xc,  -  PH'^V-^z  r'PQxo 

Since  Aq  is  arbitrary,  the  choices  of 


xc,  =  Ax^APH'^V-\z-Hx^)Ae-^PQ{x^-xo\x,{Q>)  =  x{^)  (28) 

P  =  APaPA^ATWT'^-P{H'^V-'^H-9-^Q)P,  P(0)  =  Po.  (29) 

guarantee  the  satisfaction  of  (27).  Add  the  zero  identity 

5l|A;(o)fp.  -  5||a;(()1|J,,„  +  \  J‘  ^[{kVpkW  =  0 

to  Jq  as  in  (24).  Thus 
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Jo  =  J  -  iollj  -  ||2  -  HxJ\l.,dT.  (30) 

Now  consider  the  minimization  problem 

mmi(,Jo(xo,»co) 

subject  to  constraints  (28)  and  (29).  Note  that  the  advantage  of  perturbation  method 
is  exploited  here  to  obtain  the  optimal  xq  by  neglecting  first  and  higher  order  terms 
of  the  expansion  of  the  cost  function.  Thus,  using  the  Lagrange  Multiplier  technique, 
the  first  order  optimality  condition  is  obtained  as 

0  =  Q{xc-io)  +  QPC  (31) 

C  =  -A^C  +  H^V-^HPC-e-'^QPC-e-'^Q{xc,-xo)-H^V-\z-Hxco), 

at)  =  0  (32) 

The  result  of  substitution  of  the  boundary  condition  ^(t)  =  0  into  (31)  is  subsequently 
substituted  into  (28).  Thus 

i;  =  Ax;  +  PH^V-^[z  -  Hxl],  xS(0)  =  i(0),  (33) 

where  P  is  determined  from  (29).  Note  that  (33)  is  obtained  using  the  fact  that  the 
current  time  t  is  arbitrary.  Equations  (33)  and  (29)  form  the  linear  Hoo  estimator. 
Note  that  the  worst  case  strategies  for  xq  and  Aq  are  denoted  as  Xq  and  Ag  which  are 
calculated  from 

r  A  TWT^  1  [  1  ^  f  0 

-  e-'^Q)  -A^  J  I  J  +  e-'^Qxo  _ 

x;(0)  =  m  +  PoKiOy,  Kit)  =  0  (34) 

Using  xj,  the  minimax  strategies  of  xo  and  Ao  are  denoted  as  x'^  and  A^^  which  are 
determined  from 

A  TWT^  1  f  1  .  [  0 

-  9-^Q)  -A'^  J  [  ^0^  J  L  -H'^V-^z  +  0-^Qxl 

X^{0)  =  m  +  Po^oi0y.Kit)  =  0.  (35) 

IV.  First  and  Higher  Order  Correction  Terms 

IV.l  First  Order  Correction  Term 
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To  obtain  the  first  order  correction  term  for  the  approximate  estimation  scheme, 
we  first  solve  for  worst  case  a;i  and  Ai  from  the  term  in  (18).  After  that,  the 
correction  term  is  obtained  from  the  minimization  of  J2  with  respect  to  X\  subject 
to  its  associated  constraints.  Note  that  the  first  order  necessary  condition  of  the 
minimization  problem  of  J\  with  respect  to  Xi  is  equivalent  to  the  first  order  necessary 
condition  of  the  minimization  problem  of  Jq  with  respect  to  xq.  Using  the  sweep 
method  [5],  assume 


x\  =  rcc,  +  PAJ.  (36) 

Differentiation  of  both  sides  of  (36)  yields 

[-P  +  AP  +  PA’’  +  riur’’  -  p{h'^v-'^h  -  e-'^Q)P]x; 

=  i„-Ax„+PH^V-'K  +  Hx„)-e-'PQ(x„-x,)-P{hl,fV-'(z-Hxl) 

-9;  -  PislXK 

where  (18)  is  used.  Choosing 


X,,  =  Axe,  -  PH^V-\hl  +  +  r'pg(x,,  -  Xi)  +  P{Kfv-\z  -  Hxl) 

+^S  +  P(5:ofA5,x,.(0)  =  0  (37) 

p  =  AP  +  PA’’  +  riur’’-p(p’’u-’p-r^(5)p,  p(o)  =  Po  (38) 

renders  (36)  an  identity  and  the  boundary  condition  of  (18)  satisfied.  Before  we 
proceed  further,  add  the  zero  identity 


0  =  9|a;(0)1’-p„a;(0)  -  »|A;(()|’'P(<)A;(i)  +  jIIako)!!?.,  -  jI|a:(()II?.(o 

AeJ‘^[(KfPX‘,A\{KfpyMr  (39) 

to  J2.  It  follows  that 

J2  =  efl2dT  (40) 

Jo 

where 


h  =  -  ^illo  +  *  -  hfQiXc,  -  Xz)  -  +  AJIlv-i 

+ (&)’'a;i + (2  -  Hx^fv-\Hx,,  +  />;) 

+(A;)’’Pi(A:j’'F-’(/fx;  +  a;)  -  (a:.)’’v-'(z  -  hxD 
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after  some  algebra.  Next,  assuming  that  the  zeroth-order  term  of  the  state  estimate 
vector  and  the  zeroth-order  adversaries  play  optimally, 


J2  =  J2{Xi,X2,Xo,,Xc^,x\) 

=  »  -  i.lfe  -  -  Hz’S) 

+(9S)’'A?)  +  (j  -  Hx^fV-\Hx^  +  h\)  +  (XSfP[(KfV-\Hx\  +  h’^) 

-{KfV-'(z  -  -  (gZfX{  -  (j:.)’'A?]<ir  (42) 

where  the  arguments  of  J2  of  are  written  out  explicitly  to  avoid  confusion.  In  (42), 
note  that  a;ci,a;J,Aj,  and  Xc^  are  all  driven  by  xi.  However,  from  (36),  (Eci,®!, 

Aj  are  not  completely  independent.  Thus,  it  is  sufficient  to  consider,  say,  only  the 
constraint  equations  of  x^  and  xj. 

Now,  when  the  first  order  disturbance  and  initial  state  vectors  play  their  worst 
strategies,  (18)  becomes 


xj 

[AJJ 


A  riyF 

-e-^Q)  -A^ 


f-M 

1 - 

* 

+ 


9o 


-hl^v-\z  -  Hx^)  H- 
xl(0)  =  PoA^O);  A;(<)  =  0 


(43) 


where  the  zeroth-order  term  of  the  state  estimate  vector  and  the  adversaries  are 
assumed  playing  their  minimax  strategies.  Substituting  Aj  =  P“^(xj  —  x^,)  into  the 
dynamics  of  xj  yields 


x\  =  ^4x1  -f  TWT^p-\xl 
After  minimization  of  xq,  (37)  becomes 


^c4)  +  ^o”‘,^U0)  =  PoAJ(0). 


(44) 


X,,  =  Axc,-PH^V-\h’^  +  Hx,,)  +  e-'^PQ{x,,-xi)  +  P{h^yV-Hz-Hx^) 
+So”  +  ^’(fc")’'A?,x..(0)=0.  (46) 

The  dynamics  of  Xc^  is  similarly  obtained  as 

xc,  =  {A-PH^V-'^H)xc,-e-'^PQ{xc,-X2)  +  9:  +  P{KyV-^{z-Hx:^) 
-PihlfV-^h'^  +  Hxl)  +  PiglfX'^  +  PigZfK  -  PH^V-X, 
XcAO)  =  0  (46) 
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Next,  consider  the  following  minimization  problem 


J2{x\ ,  X2,Xci ,  ,  Xj)  (47) 

subject  to  the  constraints  in  (46),  (44),  and  (45).  Introducing  Lagrange  Multiplier 
vectors  and  71  to  combine  the  constraint  equations  as  in  (46),  (44),  and  (45) 

with  J2  as  in  (42).  Thus,  the  first  order  optimality  conditions  from  the  minimization 
problem  (47)  are 


0  =  g(xc, -xi)  +  gP7i  (48) 

71  =  -AS  +  -  P'^g^PK 

-iCfK  -  -  Hx'S)  +  H^V-\Hxc,  +  h^),  7i(i)  =  0  (49) 

(71  =  -A^<Ti-P-'^TWT'^ai  +  {^fV-'^HP^i-{^fV-\z-Hxco) 

+[-{z  -  Hx:^fV-^^P  +  H^V-^hlP  -  {^of^P  -  P'^CP 

-(|^)"’](A  -  AS*)  +  {Kfy~\^  -  Hx^),  (7i(0)  =  0,  (50) 

/3i  =  -AS  +  H'^V-^HP^x  -  e~^QP^i  -  H'^V-^z  -  HxJ  -  e-'^Qixc,  -  x^), 
Mt)  =  O7  (51) 

where 


-m  _  ^9  I  .  -71 

9x  —  gx 


^1  .h^^?h. 

5x2'*=^"’  *  dx 


5x2 


In  deriving  (50),  we  have  used  the  identity 


f5gi(Xo  )ir  _  ,  rn\T 

I  J  \9xo)  • 


Recall  from  (17), 


=  -A^X^  +  H^V-'HPX^-e-'QPX^-H^V-'(z-Hx^) 

-r‘Q(x*  -  i;)A?(()  =  0  (54) 

Comparison  of  (51)  and  (54)  reveals  that  for  all  r 

y^{'r)  = 

After  substitution  of  (55)  into  (50), 


(55) 


(56) 


ax  =  -J^ax  -  P-^TWr'^ax,  (Ti(0)  =  0, 
where  the  zero  identity 

0=.HPl3x-{z-HxJ  +  {z-Hx'^)  (57) 

is  used  to  derive  (56).  Thus,  for  all  r 

=  0.  (58) 

Substituting  (55)  and  (58)  into  (49)  yields 


7.  =  -A^y,  +  -  KfA?  -  KfV-'(z  -  NX'S) 

+H'^V-\Hx.,  +  A”),  7,(()  =  0.  (59) 

Substituting  71  (t)  =  0  into  (48), 

Q[xc,(t)-ii(t)]  =  0  (60) 

Thus,  the  first  order  correction  term  of  the  optimal  state  estimate  is  obtained 
after  substituting  (60)  into  (45)  as 


K  =  Ai',-PH^V-'K-t-Hi'MA-P{liZfV-'(z-Hx'^)+g^+P{g2fX^,  f:(0)  =  0. 

(61) 

and  P  IS  calculated  as  in  (38).  In  (61),  and  are  defined  as  in  (12-15) 

when  Xi  plays  its  minimax  strategy  xj".  But  from  (27),  Xq  —  P~^{xl  -  x^o),  then 
substitution  into  (17),  we  obtain 


xs  =  (A  -h  riTr^p-')x;  -  rw^r^p-'x^o.  (62) 

Then  the  first  order  correction  involves  the  integration  of  a  2n- vector  [(xj)^,  (xj)^]. 
It  would  appear  that  the  dimension  should  grow  on  each  iteration  since  the  exact 
filter  is  infinite  dimensional.  However,  the  estimate  itself  using  only  zeroth-oder  and 
the  first  order  terms  is  x*  =  xj  •+  cxj. 

IV.2  Higher  Order  Correction  Terms 


Higher  order  correction  terms  for  the  approximate  estimation  scheme  are  derived 
similarly  to  the  first  order  case.  Thus  most  of  the  derivation  details  will  not  be 
shown.  Only  the  important  results  are  given.  For  n  >  2,  the  correction  terms  for  the 
approximate  game-theoretic  filter  are  derived  as 
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(63) 


AK  +  at,  -  PH^v-'K-,  +  Hi-J  -  P(h7„J^V-'(-z  +  Hx^) 

-  2  PKfv-'K^,.t  +  ^<-i-i)  +  E  f  Kf-'-.-i. 

:=:0  »=0 

i;(o)  =  0 


where  P  is  same  as  in  (38)  and  x*  =  Xc„  +  PKi  derive  (63).  In 

deriving  (63),  the  expansions  of  as  in  (23)  up  to  the  order  of  magnitude  (not 
e”)  are  necessary  for  the  minimization  problem 


=  min£„{Jo  +  eJi  + - h  c^"J2n)  (64) 

subject  to  its  related  constraints.  In  (64),  for  m  =  l,...,2n 


ft  a-i 

i=0 


m— 2 


^  - A 


t=0 

m-1 


«=0 


where 


(65) 


—  Xci  i  —  0,1,..., 272, 


9xi 


i=0 

t 

+  I  =  0,  l,...,2n  —  1, 

fe=0 

K,  =0,  i  <  0. 


(66) 


(67) 

(68) 


V.  Disturbance  Attenuation  and  the  Approximate  Minimax 

Estimator 


The  approximate  game-theoretic  estimator  using  up  to  infinite  order  correction 
terms  as  derived  in  sections  III  and  IV  is  first  developed  as  a  minimax  estimator 
asympotically.  Then,  the  disturbance  attenuation  problem  as  defined  in  section  II  is 
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shown  to  be  solved  by  this  infinite  order  approximate  game-theoretic  filter.  Based  on 
these,  a  N-th  order  approximate  game-theoretic  estimator  is  then  proved  to  satisfy  a 
disturbance  attenuation  inequality  with  a  higher  disturbance  attenuation  threshold. 
To  begin,  J*  as  in  (23)  is  rewritten  as 

J*  =  Jo(xo,a:o(0),u;S)  +  eJi(xi,xo,a:t(0),Xo(0),u;J,uJo) 

+e^J2{x2, xi, io, a;2(0)> ^o(0)» ' 

ri  Q-\ 

-  ^  J  -2-|l(3^cO-®o)  +  c(Xci-Xi)-f  C^(Xc2-X2)-f--*-||Q 

~2\\(~^  +  HXcl)  +  +  HXc2)  +  •  •  •llv-i 

“^11(^0  ~  ^co)  +  (-{Xi  -  a:cl)  +  —  ll(g*j,(*o)+es*,(£i)+...)^^’-* 

-|-e[(xo  -  Xco)  +  e(xi  -  Xd)  -b  -fiKoi^o)  + 

[{-z  +  Hxl)  -f  €{h;  +  Hxl)  +  ...]dr.  (69) 

(69)  is  obtained  by  first  adding  up  the  component  equations  as  derived  in  (30),  (40), 
etc.  and  then  using  the  completing  the  square  technique.  Note  that  g*  and  h*  are 
tacitly  assumed  to  be  continuous  and  differentiable.  Thus,  the  cost  using  the  minimax 
strategies  derived  in  sections  III  and  IV  is 


jM 


Jo(xS, x^(0), O  -b  eJi(xI,  xS,  ir (0), xS*(0), <,  o 
-bcV2(x;,Xi,Xo,x^(0),x5^(0),x^(0),tn^,u;f,u;S')  -b  •  •  • 

^  f  +  Hxl)  -b  e{h^  -b  Hx*)  -b  +  Hxl)  +  •  •  -llv-: 

-bc[(x-  -  xS)  -b  c(xr  -  Xj)  -b  ...nhlixl)  +  eft- (xl)  -b  .  •  -fv-^ 

[{-z  -b  ^x-)  -b  e(ft-(x;)  -b  HxT)  -b  •  •  -Idr  (70) 


After  some  algebra,  (70)  is  recast  as  a  perfect  square  expression  as 


jM 


0  f  + ^^5) + <’>0  +  + '"('■r + + ■  •  -iiv--. 

-eiw-x;)  +  £(xr-i;)  +  ---||J 

+‘^\\K + < + ■  •  •)!«  -  *:) + «  -*:)+■•  -iiiv-.^ir  (71) 


where 


Assumptions: 


(1) .  5r(a:)  and  h{x)  continuous  and  infinitely  differentiable 

(2) .  A(0)  =  =  •  • .  =  0 

(3) .  The  matrix  functions  A('),r(*),  and  H{-)  are  continuous. 

Theorem  V.l: 

For  the  dynamical  system  as  given  in  (1)  and  (2),  if  Assumptions  1  and  2  are 
satisfied,  then  the  infinite-order  game-theoretic  filter  derived  in  previous  sections  solve 
the  disturbance  attenuation  problem. 

Proof  of  Theorem  V.l: 

For  c  =  0,  the  dynamical  system  (1)  and  (2)  reduces  to 

X  =  Ax  -f-  Ftc,  z  =  Hx  -f  V  (73) 

Therefore,  the  disturbance  attenuation  problem  as  defined  in  (4)  degenerates  to  its 
corresponding  linear  problem  [6,  9].  It  is  well  known  that  the  minimax  strategy, 
obtained  from  an  approach  as  used  here  for  a  linear  problem,  is  unique  when  the 
initial  state  estimate  vector  is  chosen  as  zero.  Furthermore,  from  the  estimation 
theory  for  a  nonlinear  dynamical  system  obtained  from  a  small  perturbation  of  a  linear 
dynamical  system  as  given  in  (73),  there  exists  a  sufficiently  small  perturbation  such 
that  the  minimax  estimation  strategy  is  also  unique.  Now,  since  (ic,-,  :r,-,  re,-,  z)  =  0, 
for  i  =  0, 1,2,  •  •  •,  satisfy  the  first  order  necessary  condition  as  derived  in  section  III 
and  IV,  thus  zero  is  indeed  the  minimax  trajectory  produced  by  the  unique  minimax 
strategy.  From  (71),  if  Assumptions  1  and  2  are  satisfied,  then 

J^  =  0. 

Consequently,  the  strategies  i*,xp(0),  and  wf'  derived  in  sections  III  and  IV  are 
indeed  the  minimax  estimator  asymptotically.  Naturally,  when  the  adversaries  do 
not  play  their  minimax  strategies, 

J*  —  »A)(^0’ ^0(^)5  "I”  ^o(9)>  t^^o) 

2:2(0), a:i(0),a;o(0),u;2,ti^i,i«o)  +  •  •  •  <  0.  (74) 

Alternatively,  (74)  is  recast  as 

Dai{x*,w,x{Ql))<e,  (75) 

for  all  weL2[0,t]  and  a;(0)ei2"  s.t.  («;(T),t;(T))  ^  0  for  all  rc[0,f]  and  x(0)  /  x(0), 
where  the  disturbance  attenuation  function  for  the  infinite  order  approximate  game- 
theoretic  estimator  is  defined  as  in  section  II. 
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Denote 


N  OO 

i-*  =  +  = 

i=0  »=Ar+l 

Now,  Do/(i*,iu,a:(0))  is  decomposed  as 

C.;(i*,«.,i(0))=^<«,  (77) 

c  +  a 

where 

||a:-''^x*l|JdT  (78) 

^x*fQ^x*  +  \fx%dT  (79) 

c  =  \\x{0)  -  "^^•(0)11^:  +  f  ||«;||?^-,  +  \\z  -Hx-  eh{x)\\l-,dT  (80) 

d  =  -2[x(0)  -  ^r{0)fPo-^''x*{0)  +  f  r(0)||5,-».  (81) 

Corresponding  to  the  disturbance  attenuation  problem  defined  in  section  II,  a  N- 
th  order  disturbance  attenuation  problem  for  the  N-th  order  approximate  estimator 
truncated  from  the  infinite  order  minimax  estimator  is  defined  as  to  find  such 
that 


Dafi^x*,  w,  x(0))  <e'>e>0  (82) 

for  all  weL2[0yt]  and  x(0)ei2"  such  that  (iy(T),v(T))  ^  0  for  all  Te[0,t]  and  x(0)  ^  x(0) 
and  where 


||x(0)  -  «i(0)|| 


2 

p-l 


So  Ik  -  _ 

+  Jo  ll^llw-i  +  Ik  -Jfx-  th{x)\\l_,dT 


(83) 

Not  that  ^ X*  is  the  truncated  state  estimate  vector  as  defined  in  (76).  Note  that, 
from  (78)  and  (80), 


Theorem  V.2: 


For  the  dynamical  system  as  in  (1)  and  (2),  if  the  disturbance  attenuation  problem 
(4)  is  solved  by  the  infinite  order  minimax  estimator,  then,  the  N-th  order  disturbance 
attenuation  problem  as  defined  in  (82)  is  solved  by  the  N-th  order  estimator, 
with  a  threshold  which  is  at  most  twice  of  the  original  threshold. 

Proof  of  Thm  V.2: 


From  (77),  using  the  fact  that  c  -|-  d  >  0,  thus, 


c  c  c 

In  (81),  ^:r*(0)  =  0,  therefore  c?  =  0,  (84)  is  further  reduced  to 


a 

c 


<  d-b  <  d-i- 
c 


Q  b 
c  +  d 


<26. 


Remarks  V.3; 


(84) 


(85) 

0 


Since  N  is  arbitrary,  (85)  shows  that  the  disturbance  attenuation  inequality  is 
always  bounded  by  26  no  matter  how  many  terms  are  used  in  the  approximate  game- 
theoretic  filter. 


❖ 


Theorem  V.4: 

For  the  dynamical  system  as  given  in  (1)  and  (2),  if  the  disturbance  attenuation 
problem  is  solved  by  the  infinite  order  minimax  estimator  and  assumption  (3)  is 
satisfied,  then  the  N-th  order  disturbance  attenuation  problem  (82)  is  sloved  by  the 
N-th  order  approximate  estimator.  Moreover,  the  upper  bound  of  the  threshold  is 
proportional  to  N  -[■  1  power  of  e. 

Proof  of  Theorem  V.4: 

Under  assumption  3,  one  can  show  that  there  exists  0  <  /fi  <  oo  s.t. 


I&l  <  (86) 

where  b  is  defined  as  in  (79).  From  (80),  c  is  a  zeroth-order  term.  From  Theorem 
V.2,  ^  <6.  Therefore,  there  exists  such  that  ^  <  K2.  Thus 


c  c 


<  K2e^-^^  <  6. 


(87) 
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❖ 

Remarks  V.5: 

As  N  —*  oo,  +  0,  thus  ^  ^  0.  Therefore,  as  AT  — »  oo, 

Daf{^x\  w,  a;(0))  =  - <0+1^ -y  9.  (88) 

c  c 

This  recovers  the  infinite  order  threshold  9. 

0 

VI.  Extension  to  the  Measurement  Feedback  Control  Case 

An  extension  of  the  approximate  game-theoretic  estimator  as  derived  in  sections 
III  and  IV  is  presented  here.  In  [9],  a  linear  game- theoretic  controller  is  derived  via 
solving  a  disturbance  attenuation  problem.  To  generalize,  consider  a  small  perturba¬ 
tion  of  the  dynamical  system  in  [9]  as 


X  =  Ax  +  Bu-\-Tw  ef{x) 
z  =  Hx  +  FiU  -I-  eq{x) 

First,  a  disturbance  attenuation  function  is  defined  as 

Il^(r)ll2.,  +  Jo  11*11^  +  \HVr 


D„/(u,v,w,  i(0)) 


(89) 

(90) 


(91) 


k(0)  -  4(0)115,.,  +  +  ||»||5,_.<Jr 

Based  on  the  definition  of  (91),  the  control  disturbance  attenuation  problem  is  defined 
as  to  find  an  approximate  optimal  ocntroller  u*  s.t. 


Da/(u*,v,w,x(0))  <  9,  9  >  0,  (92) 

for  all  w,  uei/2[0,  T]  and  x(0)e/?"  such  that  (w(t),  v(t))  ^  0  for  all  re[0,  T]  and  x(0)  ^ 
x(0),  simultaneously. 

Similarly  as  in  [9],  to  obatin  a  measurement  feedback  control  based  on  causal  mapping 
of  the  measurement  sequence,  consider  the  minimax  optimization  problem 

minumaXvmaXju,x(o)J  ^  0  (93) 

subject  to  (89)  and  (90)  where 


J  = 


-jl|i(o)-x(o)|i5,..  +  i||x(r)||J, 

+l  [  Iklll  +  llxllfi  -  ^(IbllJ.-.  +  llolli'-.  )*■ 


(94) 
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In  (94),  T  is  the  terminal  time  which  is  different  from  the  present  time  t  in  the 
estimation  problem.  Note  also  that  the  maximization  of  v  is  not  trivial  in  this  problem 
due  to  the  unknown  measurement  sequence  in  the  future. 

First,  begin  with  the  maximization  problem  for  fixed  u  and  v 

J*  =  max^^x(Q)j  (95) 

subject  to  (89)  and  (90).  J  is  rewritten  as 

j  =  -jlW0)-i(0)|l5,-,  +  i||x(T)|||,+ 

T 

\  I  l|:x|fe  +  ll-llSi  -  -  «l|x  -  JJx  -  ‘q\\l-,dT  (96) 

where  V  =  FiVrf  (Fi  nonsingular).  Furthermore,  adjoin  (89)  to  J  using  a  Lagrange 
multiplier  p.  The  first  order  necessary  condition  for  problem  (95)  follows  easily  as 


x(o)  =  i(o)  +  A/)(o), /)(r)  =  r'gTx(r),  u;  =  iFrV 

P  =  -efJp-A^p-{H  +  eq^fV-\z-Hx-eq^)-e-^Qx.  (97) 


Next,  form  a  nonlinear  two  point  boundary  value  problem  from  (97)  and  (89)  as 


X 

P 


A  rwT^ 

{H  +  tq,YV-^H  -  0-^Q  -A'^  -  e{f,f 
r  ef  -\-  Bu 

+  [  -{H  +  eq,fV-\z  -  eq)  \  ^ 

a:(0)  =  i(0)  +  A/3(0),  p{T)  -  B~'^Qtx{T). 


(98) 


To  obtain  an  approximate  game-theoretic  control  scheme,  expand  x,  p,  and  u  as 

oo  oo  oo 

i=o  i=o  j=o 

Also,  from  Taylor’s  series  expansion,  let 

OO  OO  OO  oo 

/  =  X  ^  X  =  X  =  X 

j=0  i=:0  j=:0  i=0 

After  substitution  of  the  expansion,  we  have  a  series  of  linear  two  point  boundary 
problem  as 
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,0.  2^0 


TWT^  1  r  xo 


A  . 


-A^  \[po\^[-HTV-'^z 
xo(0)  =  x(0)  +  Po/>o(0),  po{T)=^e-'^QTXo{T) 

■  xi  1  _  r  A  1  [  xi  1 

Pi  J  ”  [  -6-^Q  -A'^\[pi\^ 

_  /o  +  Bui 

-flpo  +  H^V-Ao  -  qlV-\z  -  Hxo)  . 

Xi(0)  =  Popi(O),  Pi(r)  =  0~'^QtXi{T) 


(101) 


(102) 


X2 

A 

riFF  ■ 

X2 

.  ^2  . 

.  ^2  . 

-e-'^Q  -A^  J  [  />2  J 

_  /l  +  Bu2 

qly~^{Hxx  +  go)  -  qly~^{z  -Hxo)-  fjpi  -  /Jpo  + 


2:2(0)  =  -Pop2(0),  p2{T)  =  0  ^QtX2{T) 


(103) 


Furthermore,  let  Xj  and  pj,  called  the  worst  case  jth  order  state  and  Lagrange 
multiplier,  be  the  solution  to  the  corresponding  problem  in  (103).  Assume 


Xj  =  Xj  +  Pp*j,  j  =  0, 1, 2, . 

Based  on  x’j  and  Pj,  J*  is  expanded  as 


(104) 


^00  -  00  -  .J  00 

r  =  -5llE^;W'’llft  +  5llE^Kr)ei^.  +  5/ 

j=o  i=0  i=0 

00  00  00  00 

+11 E  -  «ii  E  o'AIwtt  -  -  Jf(E  -  <EfAiiUiiie) 

j=0  j=0  j=0  j=0 

where  the  expansion  series 


j=o  j=o  i=o 

are  used.  Denote  J*  =  Thus 


(106) 


■l\\pm\\%+^\K{T)rQ,+ 


2  J  II^oIIq  +  II^oIIh  ~  ^ll/’ollrw^r^'  “  “  Bx^W^^idr, 
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J;  =  -«W(o)]^ft^;(o)  +  K(o)|^QTi:(r)  + 

f(xlfQx\  +  ulRu,  -  6(flfTWT'^ p\  -  0(z  -  HxlfV-'{-Hx\  -  (h)dT, 
Jo 

n  =  j||/.:(0)|iS^  +  K(r))^<?rx;(T)  +  i||*^ 

+  j\xlfQxl  +  i||x;||^  +  ulRa^  +  i||u.||i  -  »{plfTWl'^pl  - 

«IIZ':ilfH-rx  -  Hx  -  HxlfV-'{-Hx-  -  ,;)  -  +  qlWl-.dr.  (107) 

VI.  1  Zeroth-order  Solution  -  the  Linear  Game-theoretic  Controller 

Consider  the  c°  term  of  (103).  As  in  the  estimation  problem  previlously,  we  obtain 
from  sweep  method  using  the  assumption  (104) 


io  =  Axo  +  Buo-\-pH'^V-\z-Hxa)-\-e-^PQxQ,  xo(0)  =  x(0)  (108) 

P  =  AP  +  PA^ +  TWT'^ -PiH^V-^H -e-^Q)P.  (109) 

To  determine  the  zerith-order  term  of  the  approximate  game-theoretic  control,  add 
the  zero  identity 

0  =  5[z-;(0)fPo/>;(0)  -  jlz>:(r)|^P(r)z,;{7')  +  5  (iio) 

to  Jq  as  in  (107).  Thus 

j;  =  5l|io(J’)|||,  +  PollJ  +  ||«„||=„  -  «||x  -  Hx4l.,dr  (111) 
where  St  can  be  evaluated  from 


ipo(r)|||^  =  -  (r)rt(r)  +  l||x;(r)|i|,  (112) 

St  =  QtU-0-'P(T)Qt]-'-  (113) 

Now,  consider  the  minimax  game  problem 

mmuomoxio  Jq  (114) 

subject  to  (108)  where  vq  =  z  —  Hxq.  For  conveniences,  (108)  is  rewritten  as 

xo  =  Axo  + Buo  +  fvo  (115) 
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where 


A  =  A  +  e-^PQ,  f  =  PH'^V-\  (116) 

Obviously,  from  [8],  the  zeroth-order  correction  of  the  approximate  game-theoretic 
measurement  feedback  controller  is 


ul  =  -R-^B^Sxo  (117) 

v;  =  e-^vf^Sxo  =  e-'^HPSxo  (iis) 

where  S  is  determined  from 


-S  =  SAAA^S-S{BR-^B'^-e-^fVf^)S  +  Q 

=  S{A  -f-  e-'^PQ)  +  {A  +  e-'^PQfS  -  S{BR-^B‘^  -  e-^PH^V-^HP)S  -1-  Q 
S{T)  =  St  (119) 

VI.2  First  Order  Correction  Term  of  the  Game-theoretic  Controller 

Consider  the  term  of  (103).  As  previously  in  the  estimation  problem,  using 
sweep  method,  the  first  order  correction  term  of  the  approximate  game-theoretic 
controller  is  derived  via  the  minimax  game  problem  of  with  respect  to  the  first 
order  correction  terms  of  the  controller  and  residual.  Note  that  as  previously  in  the 
estimation  problem  the  minimax  game  problem  of  J,*  with  respect  to  the  first  order 
terms  of  the  controller  and  residual  is  not  significant  for  the  similar  reason.  First, 
the  first  order  estimation  correction  term  of  the  approximate  measurement  feedback 
controller  is  derived  as 


+HfZfp'S,  ii(0)  =  0,  (120) 

P  =  AP  +  PA^ -{-TWT'^ -P(H'^V-'H-r^Q)P,  P(0)  =  Po.  (121) 

To  proceed,  after  substituting  Uq  and  Vq  into  as  in  (107)  and  using  an  appro¬ 
priately  chosen  zero  identity  as  before,  J2  is  converted  to 

Jj«,«;)  =  (KV{T)STi2(T)-¥^\\MT)\\sr+« I  I^dr  (122) 


where 
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s 


h  =  +  r‘(u;f  fii/2  +  (z  -  Hiifv-'(Hx-,  + ,;)  +  l,i>^fPl(qZfv-' 

(,r + Hx-;)  -  (g:fv-'(z  -  Hxz)  -  uzfp-\<  -  i.)  - 

-  511^*' + «”"p- 

-(x;  -  -  Hx^)  +  [ZfpZ]  (123) 

To  obtain  the  first  order  correction  term  of  the  approximate  control,  consider  the 
minimization  problem 

mm„j  J2(xi,i2,wi,  W2,a^t)  (124) 

subject  to 

x\  =  Axi  +  Bui  +  vo\,  ii(0)  =  0,  (125) 

X2  =  Ax2  +  Bu2  +  V02,  ^2(0)  =  0,  (126) 

xt  =  Axl  +  TWT^p-\xl-xi)  +  Bui  +  f^,  (127) 

x*,iT)  =  [I-0-^^PiT)QT]-^x^{T)  (128) 

where 


A  =  A  +  e-'^PQ-PH^V-'^H,  (129) 

Soi  =  -PH'^V-^q^  +  P(qZfV-\z-Hx^)  +  /S‘  +  P(/Zfp^  (130) 
«02  =  Pl-H'^V-W,+{q;yV-'(z-HxZ)-iqZfV-'(Hx-,+qZ) 

+(f:,fpz+uzfpi]+f:  (131) 

Next,  using  Lagrange  Multiplier  Techniques,  the  1st  order  optimality  condition 
for  the  problem  (124)  is  derived  as 


-A'^^  -  P-^TWT'^^  -  [{z  -  Hx'^fV-'^^P  - 

UX-^ 

+Wf^P  +  P-'fZP\i0  -  pZ)  + 

-Kqzfv-'{z-Hx^)-{^fc-nfzfpz,  (132) 

-  e-'Qx,  -  (fZfpZ  -  (qZfV-\z  -  HxZ)  +  P-'TWT'^0 

AP-'fZP(0  -  Po)  +  H'^V-\Hx,  +  ,„"),  7(T)  =  r'STii(T),  (133) 


-A^a-0-'Qxl-H‘^V-Hl,  a(T)  =  0-^STil(T),  (134) 


-««-'B1'(^  +  7), 


(135) 
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FVom  (101), 


p"  =  -A^p’S  -  e-'Qx„  -  P’S(T)  =  r'SriKT).  (136) 

Comparison  of  (134)  and  (136)  reveals  that  p^(t)  =  &{t),  for  all  r.  In  addition,  note 
that 


r^/r(^o‘)iT 


dxl 


Y  =  ifzr,  [ 


dq;{x^) 

dxl 


)"■  =  (ilf-,  -  v'-'c:  =  -V'Hx  -  HO.  (137) 


Substitution  of  (137)  into  (132)  yields 

'0=-A^^-P-'^TWT^^,  ^{0)  =  ${T)  =  0.  (138) 

Thus,  P{t)  =  0,  for  all  t.  Clearly,  (133)  is  reduced  to 


^  =  _  A’'7  -  e-'Qx,  -  UZfp"  -  (^TV-^z  -  HO  +  H^V-\Hx,  +  ,„”■).  (139) 
From  (102),  after  some  algebra. 


p~  =  -A^p’^  +  (H^V-'H-e-'Q)x\AH^V-\^-(q^fV-\z-Hx^) 

-ifZfpT,  pTm  =  r'Srilin  (140) 

Comparison  of  (139)  and  (140)  reveals  that  for  all  r, 

nr)  =  pTir).  (141) 

Substitution  of  ^{t)  =  0  and  (141)  into  (135)  yields 

„•  =  -OR-^B'^pT  =  -0R-^B^p-\x^  -  ij),  (142) 

where  P  is  calculated  from  (121).  Unfortunately,  (142)  is  not  implementable  since 
both  x^  and  are  required  to  calculate  uj. 

Alternatively,  form  a  two-point  boundary  value  problem  from  (125)  and  (139)  as 

A  -0BR-^B^  1  f  1  j. 

H'^V-^H-e-^Q  J  L  7  J 

H^V-'q^  -  UZfpS  -  (0’'t"-'(^  -  J 

0,  7(r)  =  r'Sji.(r).  (144) 


Xl  _ 

^  J 

ii(0)  = 
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As  previously,  let  the  solution  of  (143)  and  (144)  be  xj  and  7*.  Using  sweep  method, 
assume 

=  (145) 

The  choices  of 

-S  =  SA-^A^S-SBR-^B'^S-e{H'^V-'^H-6-^Q\  S(T)  =  St  (146) 
xx  =  Axi  +  uoi,  xi(T)  =  0  (147) 

render  (145)  as  an  identity,  where 

A  =  AaO-'^MQ-  M  =  P  +  05-^  (148) 

v„  =  +  M(qZfV-\z  -  Hx^)  +  M{fZf  +  /S'.  (149) 

Finally,  the  first  order  correction  terms  of  the  approximate  game-theoretic  controller 
and  the  residual  are  derived  as 

==  -R-^B'^S{xl  -  xi)  (150) 

Note  that  in  (147)  boundary  condition  is  at  the  terminal  time.  Therefore,  in  order  to 
implement  for  the  time  interval  from  the  present  time  to  future  we  need  to  rewrite 
z  —  Hx'q  as 

z-Hx'^  =  vIAH(xI-x^),  (151) 

due  to  the  fact  that  the  measurement  vector  in  the  future  is  not  available  to  the 
designer. 

VI. 3  Higher  Order  Correction  Terms  of  the  Approximate  Game-theoretic  Controller 

Similar  to  the  first  order  case,  higher  order  correction  terms  of  the  approximate 
game-theoretic  control  and  residual  is  derived  as,  for  n  <  2,  (using  s*  =  x„-l-^.§“^7*) 

<  =  (152) 

where 

Xn  =  AXn  -I-  BUn  +  ^On,  in(0)  =  0, 

Xfi  —  AXfi  -|-  Uon,  ^n(6)  0, 
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(153) 

(154) 

(155) 


1=0 

So,  =  + Cj-i) -(/;)’■/>;-. J + (157) 

9I1  =  -2-  (158) 

Furthermore,  P  and  S  are  evaluated  from  (121)  and  (146),  respectively. 

VI.4  Disturbance  Attenuation  and  the  Approximate  Controller 

The  disturbance  attenuation  property  for  the  approximate  game-theoretic  con¬ 
troller  derived  in  subsections  VI. 2  and  VI.3  is  established.  Two  disturbance  attenu¬ 
ation  properties  are  presented  here.  First,  the  disturbance  attenuation  property  for 
the  infinite  order  controller  is  proved.  Based  on  this,  the  disturbance  attenuation 
property  for  the  finite  order  approximate  controller  is  shown.  For  the  later  case,  the 
threshold  is  first  derived  as  twice  of  the  original  threshold.  Then,  as  the  number  of 
correction  terms  increased,  this  threshold  is  proved  to  shrink  and  converge  to  the 
original  one  when  infinite  order  controller  is  used. 

The  disturbance  attenuation  property  of  the  approximate  controller  using  infinite 
correction  terms  can  be  proved  similarly  as  the  estimation  problem  in  section  V. 

To  start,  J*  as  in  (105)  is  rewritten  as 


rT  /j-l 


r  =  -\\xo{T)  +  exr{T)  +  -^-\\lr  +  ^ 


Ixo  +  exi  -I- 


\q  +  +  ewi  -f 


—fracl2\\{z  —  Hxq)  -|-  £(-^0  “  Hxi)  -f  —  HX2)  -I - |l^-i 

-e||(a;o  -  io)  +  -  ®i)  -f  •  • 

+£[(25  -  xo)  +  £{x;  -  i.  +  •  ■  f)?;  +  <  +  ■  ■  +  Hx-„)  +  £(,;  +  Hx\)  +  •  ■  •)*. 

(159) 

Suppose  Uj,  for  z  =  0, 1,  •  •  •,  play  their  minimax  strategies,  then 


illiSCr)  +  (x\(T)  + . .  .|||^  + «  /  ^||i;  +  ei;  + . .  .|| J  +  ^||„;  +  ««;  +  ..  .||| 
-/racl2||(z  -  Hxl)  +  £(-«?  -  //ij)  +  e’(-5r  -  + ' '  -lit--. 

”^11(^0^  “  ^0)  +  ~  ®i)  +  • '  ■|I(/55+£/^-i--F^-* 

+e[«  -  xl)  +  eixT  -  it)  +  •  •  -figZ  +  <  +  •  •  +  Hx'^)  +  <Qo  +  Hx^)  +  •; 
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Assumptions: 

(4) .  f{x)  and  ^(a:)  continuous  and  infinitely  differentiable 

(5) .  q{0)  =  9.(0)  =  •  ■ .  =  0 

Theorem  VLl: 

For  the  Dynamical  system  as  given  in  (89)  and  (90),  if  Assumptions  (4)  and  (5) 
are  satisfied,  then  the  infinite-order  game-theoretic  controller  derived  in  sub-sections 
VL2  and  VI. 3  solve  the  disturbance  attenuation  problem  (92). 

Proof  of  Theorem  VLl: 

The  proof  here  is  brief  since  it  is  similar  to  Theorem  V.l.  To  begin,  it  is  clear  that 
{wi^Vi,Xi,Ui,z)  =  0,  for  i  =  0, 1,2,  •  •  •,  satisfy  the  first  order  necessary  conditions  as 
derived  in  section  VI,  therefore  zero  is  indeed  the  minimax  trajectory  produced  by 
the  minimax  strategy.  Thus,  if  Assumptions  (4)  and  (5)  are  satisfied,  then 

=  0  (161) 

Consequently,  the  strategies  u*,  a:™(0),  i  =  0, 1,2,  •••,  are  indeed  the  minimax 

controller  asymptotically.  If  the  adversaries  do  not  play  their  minimax  strategies, 

J.(uo,u*,*  •  •;a;o(0),xi(0),-  •  •  •)  <  0  (162) 

Alternatively,  (162)  is  recast  as 

Dajiu'’,x{0),w,v)  <  0  (163) 

for  all  it;eX2[0,  r],  veL2[0,T]  and  x{0)eR^  s.t.  {w(t),v{t))  ^  0  for  all  Te[0,r]  and 
a;(0)  ^  x(0). 

❖ 

To  show  that  a  finite  order  approximate  controller,  truncated  from  the  infinite  or¬ 
der  controller,  achieves  the  disturbance  attenuation  property,  first,  Da/(u*,t;,t/;,a;(0)) 
is  decomposed.  To  proceed,  denote 

N  00 

=  V  =  (164) 

J=0  i=N+l 

Using  (164),  Daj{u*,v,w,x{Q))  is  rewritten  as 

Daf{u*,v,w,x{0))  =  <  0,  (165) 
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where 


a  = 

MT)\\l^+  rMl  +  furndr 

Jo 

(166) 

b  = 

r2('^u*)^/?viivii^dT 

Jo 

(167) 

c  = 

T 

llx(O)  -  4(0)115,-,  +  /  +  IIHIv-.* 

0  Jo 

(168) 

Corresponding  the  disturbance  attenuation  problem  defined  in  section  VI,  a  N-th 
order  disturbance  attenuation  problem  is  defined  as  to  find  ^u*  s.t. 

Daji^u%  V,  w,  a:(0))  <  0' >  6  >  0,  (169) 

for  all  w,veL2[0,T]  and  x(0)ei?^  such  that  (t«(T),u(T))  ^  0  for  all  Tc[0,r]  and  x(0) 
x(0),  simultaneously.  Clearly 

Dafi^u”,  V,  w,  x(0))  =  ^  (170) 

Theorem  VI.  2: 

If  the  infinite  order  disturbance  attenuation  problem  as  posed  in  (92)  is  solved, 
the  N-th  order  disturbance  attenuation  problem  is  achieved  with  a  threshold  which 
is  at  most  twice  of  the  original  threshold. 

Proof  of  Theorem  VI.2: 

From  (165)  and  (170), 

Daj{’^u\v,w,x{0))  <0+^  <  20  (171) 

c  c 

0 

Theorem  VI.3: 

For  the  dynamical  system  as  given  in  (89)  and  (90),  suppose  the  infinite-order 
disturbance  attenuation  problem  as  defined  in  (92)  is  solved  by  the  infinite-order 
minimax  controller,  then  the  N-th  order  disturbance  attenuation  inequality  as  defined 
in  (169)  is  achieved  by  ^u*.  The  upper  bound  of  the  threshold  is  shown  to  be 
proportional  to  the  (N-l-l)-th  power  of  c. 

Proof  of  Theorem  VI.3: 
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From  (167),  there  exists  0  <  Kci  <  oo  such  that 


|J|  =  r  |2(E  <‘‘'1 + II E  “^iifii* 

•'O  i=0  N+l  N+1 

<  Kcie^+\ 


(172) 

(173) 


From  (168),  c  is  a  zeroth-order  term.  Moreover,  from  Theorem  VLl,  ^  There¬ 
fore,  there  exists  Kc2  satisfying  ^  <  Kc2-  Thus, 


c  c 

As  N  -*  oo,  Kc2e’^'^^  ->  0,  thus  ^  0.  Therefore, 


DajC'u*,  V,  W,  ®(0))  = 

c  c 


(174) 

(175) 

❖ 


VII.  Conclusion 

Both  nonlinear  minimax  estimator  and  controller  are  derived  via  a  regular  per¬ 
turbation  technique  by  solving  disturbance  attenuation  problems.  The  disturbance 
attenuation  problems  are  first  converted  to  their  associated  deterministic  game  prob¬ 
lems.  Then,  adopting  a  calculus  of  variation  approach,  the  estimation  and  control 
game  problem  are  solved.  In  the  solution  processes,  both  nonlinear  two-point  bound¬ 
ary  value  problems  and  the  cost  functions  are  decomposed.  Following  the  perturbation 
techniques,  the  expansion  terms  for  the  estimator  and  the  output  feedback  controller 
are  derived  sequentially.  The  optimization  of  the  odd  expansion  terms  of  the  cost 
function  are  proved  to  be  insignificant.  Most  importantly,  using  only  finite  terms,  say 
the  zeroth-order  and  the  first  order  correction  term,  both  the  nonlinear  approximate 
estimator  and  controler  are  proved  to  have  disturbance  attenuation  property.  In  addi¬ 
tion,  these  property  is  a  priori.  Using  arbitrary  term  in  the  estimator  and  controller, 
the  disturbance  attenuation  threshold  is  first  proved  to  be  twice  of  the  original  op¬ 
timal  threshold.  As  more  terms  are  used,  the  threshold  decreases  according  to 
and  converges  to  the  original  one  when  infinite  order  is  used.  This  approach  is  demon¬ 
strated  here  as  a  very  powerful  tool  to  provide  implementable  estimation  scheme  and 
control  algorithm  in  the  output  feedback  case  for  the  class  of  linear  dynamical  sys¬ 
tems  perturbed  by  a  small  scale  of  nonlinearities  both  in  the  system  dynamics  and 
the  measurement  process. 
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